hSSKJrJr SSKJr SSKJr SSKJr SSK J r \"SS55r Sr \"S 5r \S :Xa\ "\ 5r\R!5SrS S S S.r\"S\S9r\R)\ \R*/SQS9r\R,(a6\R-\R.5 \ R-\R.SS9 ggg)) dataclassfield)Optional) load_dataset) ModelCard)HfArgumentParserct\rSrSr%Sr\"SSS0S9r\\S'\"SSS 0S9r \ \S '\"S SS 0S9r \ \ \S 'Srg )ScriptArgumentsa Arguments for the script. Args: push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the dataset to the Hugging Face Hub. repo_id (`str`, *optional*, defaults to `"trl-lib/tldr"`): Hugging Face repository ID to push the dataset to. dataset_num_proc (`int` or `None`, *optional*, defaults to `None`): Number of workers to use for dataset processing. Fhelpz4Whether to push the dataset to the Hugging Face Hub.)defaultmetadata push_to_hubz trl-lib/tldrz2Hugging Face repository ID to push the dataset to.repo_idNz0Number of workers to use for dataset processing.dataset_num_proc)__name__ __module__ __qualname____firstlineno____doc__rrbool__annotations__rstrrrint__static_attributes__rP/home/james-whalen/.local/lib/python3.13/site-packages/examples/datasets/tldr.pyr r si PQKNOGS',LM'hsmrr cRSnURUSUSUSS9nSUS-nX#S.$) Nz>SUBREDDIT: r/{subreddit} TITLE: {title} POST: {post} TL;DR: subreddittitlepost)r r!r" summary)prompt completion)format)exampletldr_format_strr%r&s rto_prompt_completionr*3sH\O  # #gk.B'RYJZahioap # qFwy))J 77ra3 --- tags: [trl] --- # TL;DR Dataset ## Summary The TL;DR dataset is a processed version of Reddit posts, specifically curated to train models using the [TRL library](https://github.com/huggingface/trl) for summarization tasks. It leverages the common practice on Reddit where users append "TL;DR" (Too Long; Didn't Read) summaries to lengthy posts, providing a rich source of paired text data for training summarization models. ## Data Structure - **Format**: [Standard](https://huggingface.co/docs/trl/main/dataset_formats#standard) - **Type**: [Prompt-completion](https://huggingface.co/docs/trl/main/dataset_formats#prompt-completion) Columns: - `"prompt"`: The unabridged Reddit post. - `"completion"`: The concise "TL;DR" summary appended by the author. This structure enables models to learn the relationship between detailed content and its abbreviated form, enhancing their summarization capabilities. ## Generation script The script used to generate this dataset can be found [here](https://github.com/huggingface/trl/blob/main/examples/datasets/tldr.py). __main__zghttps://openaipublic.blob.core.windows.net/summarize-from-feedback/datasets/tldr_3_filtered/train.jsonlzghttps://openaipublic.blob.core.windows.net/summarize-from-feedback/datasets/tldr_3_filtered/valid.jsonlzfhttps://openaipublic.blob.core.windows.net/summarize-from-feedback/datasets/tldr_3_filtered/test.jsonl)train validationtestjson) data_files)idr r!r"r$)num_procremove_columnsdataset) repo_typeN) dataclassesrrtypingrdatasetsrhuggingface_hubr transformersrr r* model_cardrparserparse_args_into_dataclasses script_argsr0r4maprrrrrrr@s)!%)  68 6 z o .F446q9K{xJ 6j9Gkk--FG K//0{22iH%r