PyPI page
Home page
Author:
Semeru Lab
License:
MIT License
Summary:
ASTAligner is designed to align tokens from source code snippets to Abstract Syntax Tree (AST) nodes using Tree-sitter for AST generation and various HuggingFace tokenizers for language tokenization. The library supports a wide range of programming languages and Fast tokenizers, enabling precise mapping between source code elements and their AST representations.
Latest version:
1.0.2
Required dependencies:
concurrently
|
flask
|
flask-cors
|
protobuf
|
sentencepiece
|
tokenizers
|
transformers
|
tree-sitter
|
tree-sitter-cpp
|
tree-sitter-java
|
tree-sitter-python
|
tree-sitter-ruby
|
tree_sitter_c
|
tree_sitter_c_sharp
|
tree_sitter_go
|
tree_sitter_haskell
|
tree_sitter_html
|
tree_sitter_javascript
|
tree_sitter_kotlin
|
tree_sitter_rust
Downloads last day:
7
Downloads last week:
38
Downloads last month:
66