D_iW2SrSSKJr SSKrSSKrSSKJrJr SSKJrJ r J r SSK J r SSK Jr SSKJr SS KJr \(a SS KJr SS KJr "S S \5r"SS\ 5r"SS\5r"SS\ 5r"SS\ SS9r\"SS9"SS\55rg)zBase classes for indexing.) annotationsN)ABCabstractmethod) TYPE_CHECKINGAny TypedDict)override)beta) BaseRetriever)run_in_executor)Sequence)Documentc\rSrSrSrSSjr\SSj5r\SSj5r\SSj5r \SSj5r \SSS .SS jj5r \SSS .SS jj5r \SS j5r \SS j5r\SSSSS.SSjj5r\SSSSS.SSjj5r\SSj5r\SSj5rSrg) RecordManageraOAbstract base class representing the interface for a record manager. The record manager abstraction is used by the langchain indexing API. The record manager keeps track of which documents have been written into a `VectorStore` and when they were written. The indexing API computes hashes for each document and stores the hash together with the write time and the source id in the record manager. On subsequent indexing runs, the indexing API can check the record manager to determine which documents have already been indexed and which have not. This allows the indexing API to avoid re-indexing documents that have already been indexed, and to only index new documents. The main benefit of this abstraction is that it works across many vectorstores. To be supported, a `VectorStore` needs to only support the ability to add and delete documents by ID. Using the record manager, the indexing API will be able to delete outdated documents and avoid redundant indexing of documents that have already been indexed. The main constraints of this abstraction are: 1. It relies on the time-stamps to determine which documents have been indexed and which have not. This means that the time-stamps must be monotonically increasing. The timestamp should be the timestamp as measured by the server to minimize issues. 2. The record manager is currently implemented separately from the vectorstore, which means that the overall system becomes distributed and may create issues with consistency. For example, writing to record manager succeeds, but corresponding writing to `VectorStore` fails. cXlg)z[Initialize the record manager. Args: namespace: The namespace for the record manager. N namespace)selfrs V/home/james-whalen/.local/lib/python3.13/site-packages/langchain_core/indexing/base.py__init__RecordManager.__init__9s #cg)z2Create the database schema for the record manager.Nrs r create_schemaRecordManager.create_schemaDrc # g7f)zAAsynchronously create the database schema for the record manager.Nrrs racreate_schemaRecordManager.acreate_schemaHcg)aGet the current server time as a high resolution timestamp! It's important to get this from the server to ensure a monotonic clock, otherwise there may be data loss when cleaning up old documents! Returns: The current server time as a float timestamp. Nrrs rget_timeRecordManager.get_timeLrrc # g7f)aAsynchronously get the current server time as a high resolution timestamp. It's important to get this from the server to ensure a monotonic clock, otherwise there may be data loss when cleaning up old documents! Returns: The current server time as a float timestamp. Nrrs r aget_timeRecordManager.aget_timeWr#r$N group_ids time_at_leastcg)aUpsert records into the database. Args: keys: A list of record keys to upsert. group_ids: A list of group IDs corresponding to the keys. time_at_least: Optional timestamp. Implementation can use this to optionally verify that the timestamp IS at least this time in the system that stores the data. e.g., use to validate that the time in the postgres database is equal to or larger than the given timestamp, if not raise an error. This is meant to help prevent time-drift issues since time may not be monotonically increasing! Raises: ValueError: If the length of keys doesn't match the length of group_ids. Nrrkeysr,r-s rupdateRecordManager.updatebrrc # g7f)aAsynchronously upsert records into the database. Args: keys: A list of record keys to upsert. group_ids: A list of group IDs corresponding to the keys. time_at_least: Optional timestamp. Implementation can use this to optionally verify that the timestamp IS at least this time in the system that stores the data. e.g., use to validate that the time in the postgres database is equal to or larger than the given timestamp, if not raise an error. This is meant to help prevent time-drift issues since time may not be monotonically increasing! Raises: ValueError: If the length of keys doesn't match the length of group_ids. Nrr/s raupdateRecordManager.aupdate~r#r$cg)Check if the provided keys exist in the database. Args: keys: A list of keys to check. Returns: A list of boolean values indicating the existence of each key. Nrrr0s rexistsRecordManager.existsrrc # g7f)zAsynchronously check if the provided keys exist in the database. Args: keys: A list of keys to check. Returns: A list of boolean values indicating the existence of each key. Nrr8s raexistsRecordManager.aexistsr#r$beforeafterr,limitcg)aoList records in the database based on the provided filters. Args: before: Filter to list records updated before this time. after: Filter to list records updated after this time. group_ids: Filter to list records with specific group IDs. limit: optional limit on the number of records to return. Returns: A list of keys for the matching records. Nrrr?r@r,rAs r list_keysRecordManager.list_keysrrc # g7f)a~Asynchronously list records in the database based on the provided filters. Args: before: Filter to list records updated before this time. after: Filter to list records updated after this time. group_ids: Filter to list records with specific group IDs. limit: optional limit on the number of records to return. Returns: A list of keys for the matching records. NrrCs r alist_keysRecordManager.alist_keysr#r$cgzWDelete specified records from the database. Args: keys: A list of keys to delete. Nrr8s r delete_keysRecordManager.delete_keysrrc # g7f)zfAsynchronously delete specified records from the database. Args: keys: A list of keys to delete. Nrr8s r adelete_keysRecordManager.adelete_keysr#r$rrstrreturnNonerRrSrRfloatr0 Sequence[str]r,zSequence[str | None] | Noner- float | NonerRrSr0rXrRz list[bool] r?rYr@rYr,zSequence[str] | NonerAz int | NonerR list[str]r0rXrRrS)__name__ __module__ __qualname____firstlineno____doc__rrrr!r&r)r1r4r9r<rDrGrKrN__static_attributes__rrrrrs D # #  #AAPP     26&*   /  $     6 26&*   /  $     6     $"*.     (       ( $"*.     (       (    rrc*\rSrSr%S\S'S\S'Srg)_Recordz str | Nonegroup_idrV updated_atrN)r^r_r`ra__annotations__rcrrrreresrrec,^\rSrSrSrSU4SjjrSSjrSSjr\SSj5r \SSj5r SSS .SS jjr SSS .SS jjr SS jr SS jrSSSSS.SSjjrSSSSS.SSjjrSSjrSSjrSrU=r$)InMemoryRecordManagerz1An in-memory record manager for testing purposes.c>>[TU]U5 0UlXlg)zeInitialize the in-memory record manager. Args: namespace: The namespace for the record manager. N)superrrecordsr)rr __class__s rrInMemoryRecordManager.__init__s #,. "rcgzJIn-memory schema creation is simply ensuring the structure is initialized.Nrrs rr#InMemoryRecordManager.create_schemarrc # g7frsrrs rr!$InMemoryRecordManager.acreate_schemar#r$c,[R"5$N)timers rr&InMemoryRecordManager.get_timesyy{rc*# UR5$7frx)r&rs rr)InMemoryRecordManager.aget_time s}}Nr+c*U(a%[U5[U5:wa Sn[U5e[U5HXupVU(aX%OSnU(a X0R5:a Sn[U5eXpR5S.URU'MZ g)aUpsert records into the database. Args: keys: A list of record keys to upsert. group_ids: A list of group IDs corresponding to the keys. time_at_least: Optional timestamp. Implementation can use this to optionally verify that the timestamp IS at least this time in the system that stores. E.g., use to validate that the time in the postgres database is equal to or larger than the given timestamp, if not raise an error. This is meant to help prevent time-drift issues since time may not be monotonically increasing! Raises: ValueError: If the length of keys doesn't match the length of group ids. ValueError: If time_at_least is in the future. z-Length of keys must match length of group_idsNz!time_at_least must be in the past)rgrh)len ValueError enumerater&ro)rr0r,r-msgindexkeyrgs rr1InMemoryRecordManager.update su6 Tc)n4ACS/ !#D/JE+4y'$H!@9 o%-5]]_ UDLL  *rc,# URXUS9 g7f)aQAsync upsert records into the database. Args: keys: A list of record keys to upsert. group_ids: A list of group IDs corresponding to the keys. time_at_least: Optional timestamp. Implementation can use this to optionally verify that the timestamp IS at least this time in the system that stores. E.g., use to validate that the time in the postgres database is equal to or larger than the given timestamp, if not raise an error. This is meant to help prevent time-drift issues since time may not be monotonically increasing! r+N)r1r/s rr4InMemoryRecordManager.aupdate2s, D] KcJUVs/sHo"UR;PM sn$s snf)r7rorr0rs rr9InMemoryRecordManager.existsJs#044tt||#t444s c,# URU5$7f)zAsync check if the provided keys exist in the database. Args: keys: A list of keys to check. Returns: A list of boolean values indicating the existence of each key. )r9r8s rr<InMemoryRecordManager.aexistsUs{{4  rr>c/nURR5HLupgU(a USU:aMU(a USU::aM)U(a USU;aM;URU5 MN U(aUSU$U$)asList records in the database based on the provided filters. Args: before: Filter to list records updated before this time. after: Filter to list records updated after this time. group_ids: Filter to list records with specific group IDs. limit: optional limit on the number of records to return. Returns: A list of keys for the matching records. rhrgN)roitemsappend)rr?r@r,rAresultrdatas rrDInMemoryRecordManager.list_keys`sz.++-IC$|,6l+u4T*-Y> MM# . &5> ! rc*# URXX4S9$7f)ayAsync list records in the database based on the provided filters. Args: before: Filter to list records updated before this time. after: Filter to list records updated after this time. group_ids: Filter to list records with specific group IDs. limit: optional limit on the number of records to return. Returns: A list of keys for the matching records. r>)rDrCs rrG InMemoryRecordManager.alist_keyss".~~)  r}cRUH!nX R;dMURU M# grJrrs rrK!InMemoryRecordManager.delete_keyss$ Cll"LL%rc.# URU5 g7f)z]Async delete specified records from the database. Args: keys: A list of keys to delete. N)rKr8s rrN"InMemoryRecordManager.adelete_keyss s)rrorPrTrUrWrZr[r])r^r_r`rarbrrr!r r&r)r1r4r9r<rDrGrKrNrc __classcell__)rps@rrkrksT; #YY26&* #V#V/ #V $ #V  #VR26&* LL/ L $ L  L0 5 ! $"*. "" " ( "  " "N $"*.     (      6&rrkc0\rSrSr%SrS\S'S\S'Srg)UpsertResponseiaA generic response for upsert operations. The upsert response will be used by abstractions that implement an upsert operation for content that can be upserted by ID. Upsert APIs that accept inputs with IDs and generate IDs internally will return a response that includes the IDs that succeeded and the IDs that failed. If there are no failures, the failed list will be empty, and the order of the IDs in the succeeded list will match the order of the input documents. If there are failures, the response becomes ill defined, and a user of the API cannot determine which generated ID corresponds to which input document. It is recommended for users explicitly attach the IDs to the items being indexed to avoid this issue. r\ succeededfailedrNr^r_r`rarbrircrrrrrs&1 'rrcH\rSrSr%SrS\S'S\S'S\S'S\S'S rg ) DeleteResponseizA generic response for delete operation. The fields in this response are optional and whether the `VectorStore` returns them or not is up to the implementation. int num_deletedrXrr num_failedrNrrrrrrs8   O8rrF)totalz6Added in 0.2.29. The abstraction is subject to change.)messagec\rSrSrSr\R S Sj5rS Sjr\R S S Sjj5r S S Sjjr \R SSj5r SS jr S r g) DocumentIndexiaA document retriever that supports indexing operations. This indexing interface is designed to be a generic abstraction for storing and querying documents that has an ID and metadata associated with it. The interface is designed to be agnostic to the underlying implementation of the indexing system. The interface is designed to support the following operations: 1. Storing document in the index. 2. Fetching document by ID. 3. Searching for document using a query. c g)aUpsert documents into the index. The upsert functionality should utilize the ID field of the content object if it is provided. If the ID is not provided, the upsert method is free to generate an ID for the content. When an ID is specified and the content already exists in the `VectorStore`, the upsert method should update the content with the new data. If the content does not exist, the upsert method should add the item to the `VectorStore`. Args: items: Sequence of documents to add to the `VectorStore`. **kwargs: Additional keyword arguments. Returns: A response object that contains the list of IDs that were successfully added or updated in the `VectorStore` and the list of IDs that failed to be added or updated. Nrrrkwargss rupsertDocumentIndex.upsertrrcN# [SURU40UD6IShvN $N7f)aAdd or update documents in the `VectorStore`. Async version of `upsert`. The upsert functionality should utilize the ID field of the item if it is provided. If the ID is not provided, the upsert method is free to generate an ID for the item. When an ID is specified and the item already exists in the `VectorStore`, the upsert method should update the item with the new data. If the item does not exist, the upsert method should add the item to the `VectorStore`. Args: items: Sequence of documents to add to the `VectorStore`. **kwargs: Additional keyword arguments. Returns: A response object that contains the list of IDs that were successfully added or updated in the `VectorStore` and the list of IDs that failed to be added or updated. N)r rrs raupsertDocumentIndex.aupserts5,%  KK       %#%Nc g)aDelete by IDs or other criteria. Calling delete without any input parameters should raise a ValueError! Args: ids: List of IDs to delete. **kwargs: Additional keyword arguments. This is up to the implementation. For example, can include an option to delete the entire index, or else issue a non-blocking delete etc. Returns: A response object that contains the list of IDs that were successfully deleted and the list of IDs that failed to be deleted. Nrridsrs rdeleteDocumentIndex.delete4rrcN# [SURU40UD6IShvN $N7f)aDelete by IDs or other criteria. Async variant. Calling adelete without any input parameters should raise a ValueError! Args: ids: List of IDs to delete. **kwargs: Additional keyword arguments. This is up to the implementation. For example, can include an option to delete the entire index. Returns: A response object that contains the list of IDs that were successfully deleted and the list of IDs that failed to be deleted. N)r rrs radeleteDocumentIndex.adeleteEs5 %  KK       rc ga%Get documents by id. Fewer documents may be returned than requested if some IDs are not found or if there are duplicated IDs. Users should not assume that the order of the returned documents matches the order of the input IDs. Instead, users should rely on the ID field of the returned documents. This method should **NOT** raise exceptions if no documents are found for some IDs. Args: ids: List of IDs to get. **kwargs: Additional keyword arguments. These are up to the implementation. Returns: List of documents that were found. Nrrs rgetDocumentIndex.get\rrcN# [SURU40UD6IShvN $N7fr)r rrs ragetDocumentIndex.agetws52%  HH       rr)rzSequence[Document]rrrRrrx)rzlist[str] | NonerrrRr)rrXrrrRzlist[Document])r^r_r`rarbabcrrrrrrrrcrrrrrs    * ' 69  :   "'+ # 69  .        4      rr)rb __future__rrryrrtypingrrrtyping_extensionsr langchain_core._apir langchain_core.retrieversr langchain_core.runnablesr collections.abcr langchain_core.documentsrrrerkrrrrrrrs " #00&$34(1R CR ji MD(Y(4!9Ye!9HFGd Md Hd r