h"HSSKJrJr SSKJr SSKJrJr SSKJ r SSK J r \"SS55r Sr \ "S 5r\S :Xa\ "\ 5r\R#5Sr\"S S S 9r\R)\ \R*\R,SS9r\R r\R0"\R2"SS95\S'\R5\5r\R7SSS9r\R8(a6\R9\R:5 \R9\R:SS9 ggg)) dataclassfield)Optional)features load_dataset) ModelCard)HfArgumentParserct\rSrSr%Sr\"SSS0S9r\\S'\"SSS 0S9r \ \S '\"S SS 0S9r \ \ \S 'Srg )ScriptArgumentsa Arguments for the script. Args: push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the dataset to the Hugging Face Hub. repo_id (`str`, *optional*, defaults to `"trl-lib/rlaif-v"`): Hugging Face repository ID to push the dataset to. dataset_num_proc (`int` or `None`, *optional*, defaults to `None`): Number of workers to use for dataset processing. Fhelpz4Whether to push the dataset to the Hugging Face Hub.)defaultmetadata push_to_hubztrl-lib/rlaif-vz2Hugging Face repository ID to push the dataset to.repo_idNz0Number of workers to use for dataset processing.dataset_num_proc)__name__ __module__ __qualname____firstlineno____doc__rrbool__annotations__rstrrrint__static_attributes__rS/home/james-whalen/.local/lib/python3.13/site-packages/examples/datasets/rlaif-v.pyr r si PQK!NOGS',LM'hsmrr chSSS0SUSS./S./nSSUS S./S./nSSUS S./S./nXS/X#S .$) z Convert prompt from "xxx" to [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "xxx"}]}] and chosen and rejected from "xxx" to [{"role": "assistant", "content": [{"type": "text", "text": "xxx"}]}]. Images are wrapped into a list. usertypeimagetextquestion)r"r$)rolecontent assistantchosenrejected)promptimagesr)r*r)exampler+r)r*s rto_conversationalr.3sp FG+k*lm nF"QYIZ0[/\] ^F$67S]K^2_1`abH)9(:f ccra> --- tags: [trl] --- # RLAIF-V Dataset ## Summary The RLAIF-V dataset is a processed version of the [openbmb/RLAIF-V-Dataset](https://huggingface.co/datasets/openbmb/RLAIF-V-Dataset#dataset-card-for-rlaif-v-dataset), specifically curated to train vision-language models using the [TRL library](https://github.com/huggingface/trl) for preference learning tasks. It contains 83,132 high-quality comparison pairs, each comprising an image and two textual descriptions: one preferred and one rejected. This dataset enables models to learn human preferences in visual contexts, enhancing their ability to generate and evaluate image captions. ## Data Structure - **Format**: [Conversational](https://huggingface.co/docs/trl/main/dataset_formats#conversational) - **Type**: [Preference](https://huggingface.co/docs/trl/main/dataset_formats#preference) Columns: - `"prompt"`: The task related to the image. - `"images"`: The image. - `"chosen"`: The preferred answer. - `"rejected"`: An alternative answer that was not preferred. This structure allows models to learn to prefer the _chosen_ response over the _rejected_ one, thereby aligning with human preferences in visual tasks. ## Generation script The script used to generate this dataset can be found [here](https://github.com/huggingface/trl/blob/main/examples/datasets/rlaif-v.py). __main__zopenbmb/RLAIF-V-Datasettrain)split)num_procremove_columnswriter_batch_sizeT)decoder,g{Gz?) test_sizer5dataset) repo_typeN) dataclassesrrtypingrdatasetsrrhuggingface_hubr transformersr r r. model_cardrparserparse_args_into_dataclasses script_argsr8mapr column_namesfSequenceImagecasttrain_test_splitrrrrrrJs1)+%)  6 d : z o .F446q9K4GDGkk--++ G A##HNN$$?@AhKll1oG&&&MGK//0{22iH'r