PyPI page
Home page
Author:
TheLimeDev (@Monster-Code)
License:
MIT
Summary:
An evaluation suite for small language models (SLMs) with objective, reproducible benchmarks
Latest version:
0.0.1
Required dependencies:
datasets
|
dill
|
evaluate
|
jinja2
|
more-itertools
|
numpy
|
pytablewriter
|
rouge-score
|
sacrebleu
|
scikit-learn
|
sqlitedict
|
tqdm
|
typing-extensions
|
word2number
Optional dependencies:
accelerate
|
aiohttp
|
antlr4-python3-runtime
|
auto-gptq
|
gptqmodel
|
ibm-watsonx-ai
|
immutabledict
|
jsonlines
|
langdetect
|
litellm
|
math-verify
|
nltk
|
numpy
|
onnxruntime
|
onnxruntime-genai
|
optimum
|
optimum-habana
|
optimum-intel
|
pandas
|
peft
|
pillow
|
pytest
|
pytest-cov
|
pytest-xdist
|
python-dotenv
|
requests
|
sentencepiece
|
slm-eval
|
sparsify
|
sympy
|
tenacity
|
tiktoken
|
torch
|
tqdm
|
trackio
|
transformers
|
vllm
|
wandb
|
zeno-client
|
zstandard
Downloads last day:
0
Downloads last week:
7
Downloads last month:
124
7-day smoothing