tokenizers with a focus on performance
Project homepage: https://pypi.org/project/tokenizers
Description
An implementation of today's most used tokenizers, with a focus on
performance.
Package information
- Ports path
- textproc/py-tokenizers
- Package architecture
- amd64
- Maintainer
- The OpenBSD ports mailing-list <ports@openbsd.org>
- Categories
- textproc, lang/python, lang/rust
- Available flavors
- None listed
- Only for architectures
- aarch64, amd64, arm, i386, mips64, mips64el, powerpc, powerpc64, riscv64, sparc64, alpha, hppa
Ports list build and test dependencies as well as runtime dependencies. pkg_add selects those needed for installation.
Direct dependencies
Runtime
Build
- textproc/oniguruma
- lang/python/3
- devel/py-build
- devel/py-installer
- devel/maturin
- devel/cargo-generate-vendor STEM->=1.1
- lang/rust
Test
Used by (2 dependency relationships)
Includes library, runtime, build and test dependencies.