PyPI page
Home page
Author:
Chen Liu
License:
Non-Commercial License — see ../LICENSE.md in the LM-Dispersion repository.
Summary:
Measure layer-wise token embedding cosine similarity, assessing the severity of embedding condensation. Concept from [ICML 2026] Dispersion loss counteracts embedding condensation and improves generalization in small language models.
Latest version:
2.0
Required dependencies:
datasets
|
matplotlib
|
nltk
|
numpy
|
torch
|
tqdm
|
transformers
Optional dependencies:
pytest
Downloads last day:
47
Downloads last week:
539
Downloads last month:
539