hSSKJrJr SSKJrJrJrJrJr SSK r \(aSSK J r \"SS55r \"SS 55rg) ) dataclassfield) TYPE_CHECKINGAnyClassVarOptionalUnionN) FeatureTypec\rSrSr%Sr\\\S'\"SSS9r \ \\S'Sr \ \\S 'Sr \ \\S '\"SSSS 9r\\S 'S rS\S\\S444SjrSrg) Translation a`Feature` for translations with fixed languages per example. Here for compatibility with tfds. Args: languages (`dict`): A dictionary for each example mapping string language codes to string translations. Example: ```python >>> # At construction time: >>> datasets.features.Translation(languages=['en', 'fr', 'de']) >>> # During data generation: >>> yield { ... 'en': 'the cat', ... 'fr': 'le chat', ... 'de': 'die katze' ... } ``` languagesNFdefaultrepriddictdtypepa_typerinitr_typec[R"[UR5Vs0sHo[R"5_M sn5$s snfN)pastructsortedrstring)selflangs W/home/james-whalen/.local/lib/python3.13/site-packages/datasets/features/translation.py__call__Translation.__call__)s7yyt~~8NO8N +8NOPPOsAreturnr cpSSKJn [UR5Vs0sH o"U"S5_M sn$s snf)z2Flatten the Translation feature into a dictionary.r )Valuer)featuresr'rr)r r'ks r"flattenTranslation.flatten,s/#,24>>,BC,Bq5?",BCCCs3)__name__ __module__ __qualname____firstlineno____doc__liststr__annotations__rrrrrrrrr#r rr*__static_attributes__r,r"r r s*Cyd7B 7!E8C=!!GXc]!}5uEE3EQD}d3 3E.FFGDr6r c\rSrSr%SrSr\\\S'Sr \\ \S'\ "SSS9r \\ \S'S r\\ \S 'Sr\\\S '\ "SSSS 9r\ \S 'SrSrSrS\S\\ S444SjrSrg)TranslationVariableLanguages3a`Feature` for translations with variable languages per example. Here for compatibility with tfds. Args: languages (`dict`): A dictionary for each example mapping string language codes to one or more string translations. The languages present may vary from example to example. Returns: - `language` or `translation` (variable-length 1D `tf.Tensor` of `tf.string`): Language codes sorted in ascending order or plain text translations, sorted to align with language codes. Example: ```python >>> # At construction time: >>> datasets.features.TranslationVariableLanguages(languages=['en', 'fr', 'de']) >>> # During data generation: >>> yield { ... 'en': 'the cat', ... 'fr': ['le chat', 'la chatte,'] ... 'de': 'die katze' ... } >>> # Tensor returned : >>> { ... 'language': ['en', 'de', 'fr', 'fr'], ... 'translation': ['the cat', 'die katze', 'la chatte', 'le chat'], ... } ``` Nr num_languagesFrrrrrrrcUR(a[[UR55OSUlUR(a[UR5UlgSUlgr)rrsetlenr:r s r" __post_init__*TranslationVariableLanguages.__post_init__\s=8<DNN 34D48NNS0r6c[R"[R"[R"55[R"[R"55S.5$)Nlanguage translation)rrlist_rr>s r"r#%TranslationVariableLanguages.__call__`s8yybhhryy{&;BHHUWU^U^U`Labccr6c *[UR5n[U5SS1:XaU$UR(aX[U5U- (aE[SSR[ [U5U- 55SSRU5S35e/nUR 5HQupE[ U[5(aURXE45 M.URUVs/sHodU4PM sn5 MS [[ U56upxXxS.$s snf)NrCrDzSome languages in example (z, z) are not in valid set (z).rB) r<r ValueErrorjoinritems isinstancer3appendextendzip) r translation_dictlang_settranslation_tuplesr!textelr translationss r"encode_example+TranslationVariableLanguages.encode_examplecst~~&  Z$? ?# # ^^$4 5 @-diisCS?TW_?_8`.a-bbz{|E|EFN|O{PPRS   *002JD$$$"))4,7"))*E"":*EF 3#&v.@'A"B %CC +Fs! D r%r cLSSKJnJn U"U"S55U"U"S55S.$)zCFlatten the TranslationVariableLanguages feature into a dictionary.r )Listr'rrB)r(rXr')r rXr's r"r*$TranslationVariableLanguages.flattenzs))U8_-h0  r6)rr:)r-r.r/r0r1rrr2r4r:intrrr3rrrrrr?r#rUr rr*r5r,r6r"r8r83s>!%Ix~$#'M8C='d7B 7!E8C=!!GXc]!=EPUVE3VMdD. }d3 3E.FFG r6r8) dataclassesrrtypingrrrrr pyarrowrr(r r r8r,r6r"r^sR(@@% $D$D $DN M M  M r6