Search results

BPE Dropout tokenizer generates unk at the beginning of sequence

Status: Open.

#1071 In google/sentencepiece;

· AnnaLebedeva opened on Dec 1, 2024

The pip command to install the SentencePiece Python module fails.

Status: Open.

#1070 In google/sentencepiece;

· tprrt opened on Nov 20, 2024

Doesn't seem to work with Python 3.13

Status: Open.

#1069 In google/sentencepiece;

· tprrt opened on Nov 20, 2024

Initialized number of seed sentencepieces too low

Status: Open.

#1068 In google/sentencepiece;

· DmitriiP20 opened on Nov 14, 2024

Asan detects memory leak in sentencepiece/_sentencepiece.cpython-312-x86_64-linux-gnu.so+0x6f7f4

Status: Open.

#1065 In google/sentencepiece;

· renxida opened on Nov 4, 2024

Compatibility Issue when using v0.2.0 with transformers and tensorflow

Status: Open.

#1060 In google/sentencepiece;

· aws-tianquaw opened on Oct 2, 2024

Crashes on out of range inputs depending on other inputs

bug

Status: Open.

#1051 In google/sentencepiece;

· colehaus opened on Sep 10, 2024

With unigram algorithm, constant piece at end of each sentences does not become a token

bug

Status: Open.

#1047 In google/sentencepiece;

· jogardi opened on Aug 29, 2024

Error Attribute Error: type object 'SentencePieceTrainer' has no attribute 'train'. Did you mean: 'Train'?

Status: Open.

#1046 In google/sentencepiece;

· bop578530 opened on Aug 23, 2024

Zero Width Joiner issue for Sinhala Language

Status: Open.

#1031 In google/sentencepiece;

· Nadil-K opened on Jun 30, 2024

No typings in Python package

enhancement

Status: Open.

#1030 In google/sentencepiece;

· marcospgp opened on Jun 29, 2024

When I set SPM_PROTOBUF_PROVIDER to "package" in CMakeLists.txt, the compilation fails.

Status: Open.

#1029 In google/sentencepiece;

· hhxdestiny opened on Jun 25, 2024