PyPI page
Home page
Author:
Adam Twardoch
License:
Apache-2.0
Summary:
A Python tool for splitting large Markdown files into smaller sections based on a specified token limit. This is particularly useful for processing large Markdown files with GPT models, as it allows the models to handle the data in manageable chunks.
Latest version:
1.0.9
Required dependencies:
fire
|
importlib-metadata
|
mistletoe
|
python-frontmatter
|
pyyaml
|
regex
|
syntok
|
tiktoken
Optional dependencies:
pytest
|
pytest-cov
|
setuptools
Downloads last day:
124
Downloads last week:
532
Downloads last month:
1,950