ȅiaSSKrSSKrSSKJrJr SSKJr SSKJr SSK J r SSK J r SSK Jr SSKJrJrJrJr \ (aSS KJr S S /r"S S \5r"SS \5r"SS \5rg)N) GeneratorSequence)contextmanager)Path) TYPE_CHECKING) url_to_fs)StreamTransformExtension)FileSystemBaseFileSystemReaderFileSystemWriterSerializationFormat)AbstractFileSystem FsspecWriter FsspecReaderc >\rSrSrSSjr\S\\R-S\S\ \ RSS44Sj5r S\\R-S\S\\R-4S jr S\\R-S\\R-4S jrS\\R-S \\R-SS4S jrS\\R-SS4S jr\S\\R-S\4Sj5rS\\R-S\4SjrS\\R-SS4SjrS\\R-S\\4SjrSrg) FileSystem returnNcSUlgNfs)selfs i/home/james-whalen/.local/lib/python3.13/site-packages/torch/distributed/checkpoint/_fsspec_filesystem.py__init__FileSystem.__init__!s -1pathmodec#V^# URc [S5e[R"U5nURR UT5nUv SSS5 g! [ U4SjS55(aUR U5 e! e=fe=f!,(df  g=f7f)Nzfs should not be Nonec3,># UH oT;v M g7fr).0chrs r +FileSystem.create_stream..3s2EbTzEszw+a)rAssertionErrorosfspathopenanyrm_file)rrrstreams ` r create_streamFileSystem.create_stream$s 77? !89 9yy WW\\$ %  & % 2E222 T*& %sHA B)BA B)B;B  B BBB B&"B)suffixc@[RRX5$r)r(rjoin)rrr0s r concat_pathFileSystem.concat_path:sww||D))rc .[U40UD6uUlnU$r)rr)rrkwargs_s r init_pathFileSystem.init_path=st.v.  rnew_pathc:URRX5 gr)rrename)rrr:s rr<FileSystem.renameAs t&rc8URRUSS9 g)NT)exist_ok)rmakedirsrrs rmkdirFileSystem.mkdirDs -r checkpoint_idch[U[5(ag[U5 g![a gf=f)NFT) isinstancerr ValueErrorclsrDs rvalidate_checkpoint_id!FileSystem.validate_checkpoint_idGs8 mT * *  m $  s $ 11c8URRU5$r)rexistsrAs rrMFileSystem.existsSsww~~d##rc:URRU5 gr)rrmrAs rr,FileSystem.rm_fileVs  4rc6URRUSS9$)NF)detail)rlsrAs rrT FileSystem.lsYswwzz$uz--rr)rN)__name__ __module__ __qualname____firstlineno__rrstrr(PathLikerioIOBaser.r3r8r<rB classmethodboolrJrMr,listrT__static_attributes__r"rrrr sh2"++%-0 299dD( )**bkk 1*3*3CT*cBKK/cBKK>O'3,'bkk8I'd'.# +.. 33D   $3,$$C"++-$.sR[[(.T#Y.rrc^\rSrSrSrSSSSSS\R 4S\\R-S\ S \ S \ S \ S \ S \ \ S-S\SS4U4Sjjjr\S\\R-S\ 4Sj5rSrU=r$)r`aD Basic implementation of StorageWriter using FFspec. This implementation makes the following assumptions and simplifications: * The checkpoint path is an empty or non-existing directory. * File creation is atomic The checkpoint consist of one file per write request plus a `.metadata` file with the serialized metadata. TiNrsingle_file_per_rank sync_files thread_countper_thread_copy_ahead overwrite _extensionsserialization_formatrc >[T U]UUUUUUUUS9 [5UlURR"U40U D6Ulg)a Initialize the writer pointing to `path`. Args: path: directory where the checkpoint will be written to. single_file_per_rank: Produce one file per rank instead of one file per tensor/blob. Default to True. sync_files : force files to be synced to permanent storage. Default to True. thread_count: Number of IO threads to use to write. Default to 1. per_thread_copy_ahead: How many bytes to copy from the GPU ahead of saving then. Default 10Mb. overwrite: Whether to allow overwriting existing checkpoints. Defaults to True. _extensions: Extensions to apply to output streams (EXPERIMENTAL) N. B. If sync_files is disabled, there's no guarantee that the checkpoint will be consistent in the case of a failure. )rirjrkNsuperrrrr8r) rrrerfrgrhrirjrkr6 __class__s rrFsspecWriter.__init__nsU4     !#!5  ,GG%%d5f5 rrDc,[RU5$rrrJrHs rrJ#FsspecWriter.validate_checkpoint_id00??rrr)rVrWrXrY__doc__r TORCH_SAVErZr(r[r_intrr rr^rJra __classcell__ros@rrr`s  &*%/AE4G4R4R%6BKK%6#%6 %6  %6 # %6%667$>%62%6 %6%6N@33D@@@rc^\rSrSrS\\R -SS4U4Sjjr\S\\R -S\ 4Sj5r Sr U=r $) rrrNc >[TU]U5 [5UlURR"U40UD6Ulgrrm)rrr6ros rrFsspecReader.__init__s4 ,GG%%d5f5 rrDc,[RU5$rrrrHs rrJ#FsspecReader.validate_checkpoint_idrtrru) rVrWrXrYrZr(r[rr^r_rJraryrzs@rrrsN6S2;;.6T6 @33D@@@r)r\r(collections.abcrr contextlibrpathlibrtypingr fsspec.corer'torch.distributed.checkpoint._extensionr 'torch.distributed.checkpoint.filesystemr r r r fsspecr__all__rrrr"rrrsl /% !L)  <.<.@7@#7@t@#@r