| Port details |
- py-sentencepiece Unsupervised text tokenizer for Neural Network-based text generation
- 0.2.0_1 textproc
 =2      0.2.0_1Version of this port present on the latest quarterly branch. - Maintainer: yuri@FreeBSD.org
- Port Added: 2023-01-25 07:48:07
- Last Update: 2025-03-08 04:05:21
- Commit Hash: 06a08e6
- People watching this port, also watch:: jdictionary, py311-Automat, py311-python-gdsii, py311-PyOpenGL, p5-Sane
- License: APACHE20
- WWW:
- https://github.com/google/sentencepiece
- Description:
- SentencePiece is an unsupervised text tokenizer and detokenizer mainly for
Neural Network-based text generation systems where the vocabulary size is
predetermined prior to the neural model training. SentencePiece implements
subword units (e.g., byte-pair-encoding (BPE)) and unigram language model
with the extension of direct training from raw sentences. SentencePiece
allows us to make a purely end-to-end system that does not depend on
language-specific pre/postprocessing.
  ¦  ¦  ¦  ¦ 
- Manual pages:
- FreshPorts has no man page information for this port.
- pkg-plist: as obtained via:
make generate-plist - There is no configure plist information for this port.
- Dependency lines:
-
- ${PYTHON_PKGNAMEPREFIX}sentencepiece>0:textproc/py-sentencepiece@${PY_FLAVOR}
- To install the port:
- cd /usr/ports/textproc/py-sentencepiece/ && make install clean
- To add the package, run one of these commands:
- pkg install textproc/py-sentencepiece
- pkg install py311-sentencepiece
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-sentencepiece listed in the above command, you can pick from the names under the Packages section.- PKGNAME: py311-sentencepiece
- Package flavors (<flavor>: <package>)
- py311: py311-sentencepiece
- distinfo:
- TIMESTAMP = 1708411814
SHA256 (google-sentencepiece-v0.2.0_GH0.tar.gz) = 9970f0a0afee1648890293321665e5b2efa04eaec9f1671fcf8048f456f5bb86
SIZE (google-sentencepiece-v0.2.0_GH0.tar.gz) = 11980811
Packages (timestamps in pop-ups are UTC):
- Dependencies
- NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
- Build dependencies:
-
- protobuf>0 : devel/protobuf
- pkgconf>=1.3.0_1 : devel/pkgconf
- py311-setuptools>=63.1.0 : devel/py-setuptools@py311
- python3.11 : lang/python311
- Test dependencies:
-
- python3.11 : lang/python311
- Runtime dependencies:
-
- python3.11 : lang/python311
- Library dependencies:
-
- libsentencepiece.so : textproc/sentencepiece
- This port is required by:
- for Run
-
- misc/py-gguf
Configuration Options:- No options to configure
- Options name:
- textproc_py-sentencepiece
- USES:
- compiler:c++17-lang pkgconfig python
- FreshPorts was unable to extract/find any pkg message
- Master Sites:
|
Number of commits found: 6
| Commit History - (may be incomplete: for full details, see links to repositories near top of page) |
| Commit | Credits | Log message |
|---|
0.2.0_1
08 Mar 2025 04:05:21
 |
Charlie Li (vishwin) |
python: bump all USE_PYTHON=distutils consumers after RUN_DEPENDS removal
Any missed ports, feel free to bump.
Any ports that need setuptools at runtime can have the devel/py-setuptools
manually added back to RUN_DEPENDS, but understand that this practice
is deprecated; see CHANGES for details. |
0.2.0
21 Feb 2024 09:15:27
|
Yuri Victorovich (yuri) |
textproc/py-sentencepiece: Fix tests |
0.2.0
20 Feb 2024 07:09:21
|
Yuri Victorovich (yuri) |
textproc/{,py-}sentencepiece: update 0.1.99 → 0.2.0
Reported by: portscout |
0.1.99
03 Jun 2023 07:43:33
|
Yuri Victorovich (yuri) |
textproc/sentencepiece: Update 0.1.98 → 0.1.99
Reported by: portscout |
0.1.98
18 Apr 2023 04:38:14
|
Yuri Victorovich (yuri) |
textproc/{,py-}sentencepiece: Update 0.1.97 → 0.1.98
Reported by: portscout |
0.1.97
25 Jan 2023 07:45:57
|
Yuri Victorovich (yuri) |
textproc/py-sentencepiece: New port: Unsupervised text tokenizer for Neural
Network-based text generation |
Number of commits found: 6
|
As an Amazon Associate I earn from qualifying purchases.