Creating a custom collation maybe is not the most common task, but you definitely need to know how to do it. If you are using 🤗 Transformers, try to write a collator that will tokenize the data on the fly. More content atplainenglish.io. Sign up for ourfree weekly newsletter. Get ...
我认为最好先调试train函数,实际上可以通过从数据集采样并整理输出来调试collate函数:
在train_dataset上只有一个示例,因此请尝试将批大小设置为1。
在train_dataset上只有一个示例,因此请尝试将批大小设置为1。