Hi. Thanks for the question and for pointing this out.

You are correct that the actual item range is from 1 to 12101. However, we include item 0 as a padding token to maintain consistent sequence lengths during precessing. For simplicity, logits are calculated for item 0 as well, but it's important to note that item 0 never appears in the target label answers. This ensures that the model focuses only on valid items in the range of 1 to 12101.

As you mentioned, it is more precise to use answers-1 and exclude the first row of logits for computing cross-entropy loss. However, we opted for our current implementation, which avoids additional adjustments to the indices while maintaining equivalent results.

I hope this clarifies your concern. If you have further questions, please let me know.

Hi. Thanks for the question and for pointing this out.
Best regards！