h%SSKJr SSKrSSKJr SSKJr SSKrSSK J r SSK J r SSK JrJr SSKJr SS KJrJr S \R*S '"S S 5r"SS5rg)) annotationsN)Path)Any)Image) IMG_FORMATS)LOGGER TORCH_VERSION)check_requirements) TORCH_2_4 select_deviceTRUEKMP_DUPLICATE_LIB_OKcX\rSrSrSrS SjrS SjrS SjrSSjrSSSjjr SSjr S r g )VisualAISearcha A semantic image search system that leverages OpenCLIP for generating high-quality image and text embeddings and FAISS for fast similarity-based retrieval. This class aligns image and text embeddings in a shared semantic space, enabling users to search large collections of images using natural language queries with high accuracy and speed. Attributes: data (str): Directory containing images. device (str): Computation device, e.g., 'cpu' or 'cuda'. faiss_index (str): Path to the FAISS index file. data_path_npy (str): Path to the numpy file storing image paths. data_dir (Path): Path object for the data directory. model: Loaded CLIP model. index: FAISS index for similarity search. image_paths (list[str]): List of image file paths. Methods: extract_image_feature: Extract CLIP embedding from an image. extract_text_feature: Extract CLIP embedding from text. load_or_build_index: Load existing FAISS index or build new one. search: Perform semantic search for similar images. Examples: Initialize and search for images >>> searcher = VisualAISearch(data="path/to/images", device="cuda") >>> results = searcher.search("a cat sitting on a chair", k=10) c V[(dS[S35eSSKJn [ S5 [ S5UlSUlSUl[URS S 55Ul [URS S 55Ul URR5(dMSS KJn ["R$"URSUS35 SSKJn U"US3SSS9 [S 5Ul U"SURS9UlSUl/UlUR15 g)zDInitialize the VisualAISearch class with FAISS index and CLIP model.z1VisualAISearch requires torch>=2.4 (found torch==)r)build_text_modelz faiss-cpufaissz faiss.indexz paths.npydataimagesdevicecpu) ASSETS_URLz( not found. Downloading images.zip from z /images.zip) safe_downloadT)urlunzipretryz clip:ViT-B/32)rN)r r ultralytics.nn.text_modelrr __import__r faiss_index data_path_npyrgetdata_dirr rexistsultralytics.utilsrrwarningultralytics.utils.downloadsrmodelindex image_pathsload_or_build_index)selfkwargsrrrs a/home/james-whalen/.local/lib/python3.13/site-packages/ultralytics/solutions/similarity_search.py__init__VisualAISearch.__init__2sy^Mm_\]^^y>;'( ((VZZ9: #FJJx$?@ }}##%% 4 NNdmm_,TU_T``kl m A  K8A N NDM%odkkJ     "cURR[R"U55R 5R 5$)z7Extract CLIP image embedding from the given image path.)r* encode_imageropenrnumpy)r.paths r0extract_image_feature$VisualAISearch.extract_image_featureOs2zz&&uzz$'78<<>DDFFr3cURRURRU/55R5R 5$)z6Extract CLIP text embedding from the given text query.)r* encode_texttokenizerr7)r.texts r0extract_text_feature#VisualAISearch.extract_text_featureSs:zz%%djj&9&94&&ABFFHNNPPr3c[UR5R5(a[UR5R5(af[R "S5 UR RUR5Ul[R"UR5Ul g[R "S5 /nURR5H|nURR5R!S5[";aM6UR%UR'U55 URR%UR(5 M~ U(d [/S5e[R0"U5R3S5nUR R5U5 UR R7UR8S 5UlURR;U5 UR R=URUR5 [R>"UR[R@"UR55 [R "S [CUR5S 35 g![*a2n[R,"SUR(SU35 SnAGMSnAff=f) aF Load existing FAISS index or build a new one from image features. Checks if FAISS index and image paths exist on disk. If found, loads them directly. Otherwise, builds a new index by extracting features from all images in the data directory, normalizes the features, and saves both the index and image paths for future use. zLoading existing FAISS index...Nz#Building FAISS index from images....z Skipping z: z'No image embeddings could be generated.float32zIndexed z images.)"rr"r&r#rinfor read_indexr+nploadr,r%iterdirsuffixlowerlstriprappendr9name Exceptionr( RuntimeErrorvstackastype normalize_L2 IndexFlatIPshapeadd write_indexsavearraylen)r.vectorsfilees r0r-"VisualAISearch.load_or_build_indexWs   ! ( ( * *tD4F4F/G/N/N/P/P KK9 :..t/?/?@DJ!wwt'9'9:D    9:MM))+D{{  "))#.kA =t99$?@  '' 2,HI I))G$++I6 (ZZ++GMM!,<=  w tzz4+;+;< ""BHHT-=-=$>? hs4#3#345X>? =499+Rs;<< =s AJ  K*&KKc ,URU5RS5nURRU5 URR XB5upV[ US5VVs/sH3upxUSUU:dMURU[USU54PM5 n nnU RSSS9 [R"S5 U H"up[R"SU SU S 35 M$ U V s/sHoSPM sn $s snnfs sn f) a Return top-k semantically similar images to the given query. Args: query (str): Natural language text query to search for. k (int, optional): Maximum number of results to return. similarity_thresh (float, optional): Minimum similarity threshold for filtering results. Returns: (list[str]): List of image filenames ranked by similarity score. Examples: Search for images matching a query >>> searcher = VisualAISearch(data="images") >>> results = searcher.search("red car", k=5, similarity_thresh=0.2) rCrc US$)NrD)xs r0'VisualAISearch.search..s1Q4r3T)keyreversez Ranked Results:z - z | Similarity: z.4f) r?rRrrSr+search enumerater,floatsortrrE) r.queryksimilarity_thresh text_featDr+idxiresultsrNscorers r0rgVisualAISearch.searchs"--e4;;IF   *::$$Y2BKERSHBU BUYZ[\Y]^aYbfwYw 3T  a %!S "2 3BU     6 '("KD KK$tfOE#;? @#&&g!g&& 's*D $D :Dc$URU5$)z.Direct call interface for the search function.)rg)r.rks r0__call__VisualAISearch.__call__s{{5!!r3)r%r#rrr"r,r+r*N)r/rreturnNone)r8rry np.ndarray)r>strryr{)ryrz)g?)rkr|rlintrmriry list[str])rkr|ryr) __name__ __module__ __qualname____firstlineno____doc__r1r9r?r-rgrw__static_attributes__rar3r0rrs*:#:GQ+@Z'@"r3rc>\rSrSrSrSS SjjrS SjrS S SjjrSrg) SearchAppa A Flask-based web interface for semantic image search with natural language queries. This class provides a clean, responsive frontend that enables users to input natural language queries and instantly view the most relevant images retrieved from the indexed database. Attributes: render_template: Flask template rendering function. request: Flask request object. searcher (VisualAISearch): Instance of the VisualAISearch class. app (Flask): Flask application instance. Methods: index: Process user queries and display search results. run: Start the Flask web application. Examples: Start a search application >>> app = SearchApp(data="path/to/images", device="cuda") >>> app.run(debug=True) Nc[S5 SSKJnJnJn X@lXPl[ XS9UlU"[S[U5R5SS9Ul URRSURS S /S 9 g ) z Initialize the SearchApp with VisualAISearch backend. Args: data (str, optional): Path to directory containing images to index and search. device (str, optional): Device to run inference on (e.g. 'cpu', 'cuda'). z flask>=3.0.1r)Flaskrender_templaterequest)rr templatesz/images)template_folder static_folderstatic_url_path/GETPOST) view_funcmethodsN) r flaskrrrrsearcherrrresolveapp add_url_ruler+)r.rrrrrs r0r1SearchApp.__init__sq >*99. &D@  't*,,.%   cTZZ%Qr3c/nURRS:XaEURRRSS5R 5nUR U5nUR SUS9$)zCProcess user query and display search results in the web interface.rrkzsimilarity-search.html)rr)rmethodformr$striprr)r.rrrks r0r+SearchApp.indexsc <<  & (LL%%))'26<<>EmmE*G##$rsK# .37B%+ !"R"R"j7"7"r3