From the SFT website:

Dataset format support

The SFTTrainer supports popular dataset formats. This allows you to pass the dataset to the trainer without any pre-processing directly. The following formats are supported:

instruction format
Copied
{"prompt": "", "completion": ""}
{"prompt": "", "completion": ""}
{"prompt": "", "completion": ""}

My local dataset is exactly structured like this, however the SFTTrainer gives me errors:

ValueError: Column to remove ['train'] not in the dataset. Current columns in the dataset: ['prompt', 'completion']

and

raise ValueError("You need to specify either text or text_target.")
ValueError: You need to specify either text or text_target.

or

value = self.data[key]
        ~~~~~~~~~^^^^^

KeyError: 'text'

Either loading it like this:

train_dataset = load_dataset('json', data_files=dataset_file_path)

or like that

train_dataset = load_dataset('json', data_files=dataset_file_path, field='train')

Adding a text key in the dataset doesn't change it, either. So, what is the expected dataset format, if it's not the one specified on the website?