PyPI page
Home page
Author:
Taishi Ikeda
Summary:
A comparison tool of Japanese tokenizers
Latest version:
0.0.11
Required dependencies:
janome
|
pandas
|
py-cpuinfo
|
requests
|
scikit-learn
|
tqdm
Optional dependencies:
catalyst
|
fugashi
|
ginza
|
ipadic
|
kytea
|
mecab-python3
|
nagisa
|
pyknp
|
sentencepiece
|
spacy
|
sudachidict_core
|
sudachipy
|
tinysegmenter3
|
torch
|
transformers
|
unidic-lite
Downloads last day:
7
Downloads last week:
15
Downloads last month:
73