notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)Want a good monitor light? See my photosAll times are UTC		 Ukraine

Newsfeed changes

The news feed page[s] were not laid out well. Two pages, disjointed information, hard to figure out how to use the optional parameters...

Thankfully, someone told me.

The new page is ready for your review. Please compare these two:

You may also be interested in the Github issue.
Port details
py-tiktoken Fast BPE tokeniser for use with OpenAI's models
0.9.0_1 textproc on this many watch lists=2 search for ports that depend on this port Find issues related to this port Report an issue related to this port View this port on Repology. pkg-fallout 0.9.0_1Version of this port present on the latest quarterly branch.
Maintainer: sunpoet@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2023-08-21 17:15:36
Last Update: 2025-02-24 07:55:52
Commit Hash: cf7c4b0
People watching this port, also watch:: jdictionary, py311-Automat, py311-python-gdsii, py311-PyOpenGL, p5-Sane
Also Listed In: python
License: MIT
WWW:
https://github.com/openai/tiktoken
Description:
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Homepage    cgit ¦ Codeberg ¦ GitHub ¦ GitLab ¦ SVNWeb - no subversion history for this port

Manual pages:
FreshPorts has no man page information for this port.
pkg-plist: as obtained via: make generate-plist
There is no configure plist information for this port.
Dependency lines:
  • ${PYTHON_PKGNAMEPREFIX}tiktoken>0:textproc/py-tiktoken@${PY_FLAVOR}
To install the port:
cd /usr/ports/textproc/py-tiktoken/ && make install clean
To add the package, run one of these commands:
  • pkg install textproc/py-tiktoken
  • pkg install py311-tiktoken
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-tiktoken listed in the above command, you can pick from the names under the Packages section.
PKGNAME: py311-tiktoken
Package flavors (<flavor>: <package>)
  • py311: py311-tiktoken
distinfo:
TIMESTAMP = 1739930208 SHA256 (tiktoken-0.9.0.tar.gz) = d02a5ca6a938e0490e1ff957bc48c8b078c88cb83977be1625b1fd8aac792c5d SIZE (tiktoken-0.9.0.tar.gz) = 35991

Expand this list (62 items)

Collapse this list.

SHA256 (rust/crates/aho-corasick-1.1.3.crate) = 8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916 SIZE (rust/crates/aho-corasick-1.1.3.crate) = 183311 SHA256 (rust/crates/autocfg-1.4.0.crate) = ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26 SIZE (rust/crates/autocfg-1.4.0.crate) = 17712 SHA256 (rust/crates/bit-set-0.5.3.crate) = 0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1 SIZE (rust/crates/bit-set-0.5.3.crate) = 14470 SHA256 (rust/crates/bit-vec-0.6.3.crate) = 349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb SIZE (rust/crates/bit-vec-0.6.3.crate) = 19927 SHA256 (rust/crates/bstr-1.11.3.crate) = 531a9155a481e2ee699d4f98f43c0ca4ff8ee1bfd55c31e9e98fb29d2b176fe0 SIZE (rust/crates/bstr-1.11.3.crate) = 351536 SHA256 (rust/crates/cfg-if-1.0.0.crate) = baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd SIZE (rust/crates/cfg-if-1.0.0.crate) = 7934 SHA256 (rust/crates/fancy-regex-0.13.0.crate) = 531e46835a22af56d1e3b66f04844bed63158bc094a628bec1d321d9b4c44bf2 SIZE (rust/crates/fancy-regex-0.13.0.crate) = 85930 SHA256 (rust/crates/heck-0.5.0.crate) = 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea SIZE (rust/crates/heck-0.5.0.crate) = 11517 SHA256 (rust/crates/indoc-2.0.5.crate) = b248f5224d1d606005e02c97f5aa4e88eeb230488bcc03bc9ca4d7991399f2b5 SIZE (rust/crates/indoc-2.0.5.crate) = 14396 SHA256 (rust/crates/libc-0.2.169.crate) = b5aba8db14291edd000dfcc4d620c7ebfb122c613afb886ca8803fa4e128a20a SIZE (rust/crates/libc-0.2.169.crate) = 757901 SHA256 (rust/crates/memchr-2.7.4.crate) = 78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3 SIZE (rust/crates/memchr-2.7.4.crate) = 96670 SHA256 (rust/crates/memoffset-0.9.1.crate) = 488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a SIZE (rust/crates/memoffset-0.9.1.crate) = 9032 SHA256 (rust/crates/once_cell-1.20.3.crate) = 945462a4b81e43c4e3ba96bd7b49d834c6f61198356aa858733bc4acf3cbe62e SIZE (rust/crates/once_cell-1.20.3.crate) = 33456 SHA256 (rust/crates/portable-atomic-1.10.0.crate) = 280dc24453071f1b63954171985a0b0d30058d287960968b9b2aca264c8d4ee6 SIZE (rust/crates/portable-atomic-1.10.0.crate) = 174760 SHA256 (rust/crates/proc-macro2-1.0.93.crate) = 60946a68e5f9d28b0dc1c21bb8a97ee7d018a8b322fa57838ba31cc878e22d99 SIZE (rust/crates/proc-macro2-1.0.93.crate) = 52388 SHA256 (rust/crates/pyo3-0.22.6.crate) = f402062616ab18202ae8319da13fa4279883a2b8a9d9f83f20dbade813ce1884 SIZE (rust/crates/pyo3-0.22.6.crate) = 546746 SHA256 (rust/crates/pyo3-build-config-0.22.6.crate) = b14b5775b5ff446dd1056212d778012cbe8a0fbffd368029fd9e25b514479c38 SIZE (rust/crates/pyo3-build-config-0.22.6.crate) = 31369 SHA256 (rust/crates/pyo3-ffi-0.22.6.crate) = 9ab5bcf04a2cdcbb50c7d6105de943f543f9ed92af55818fd17b660390fc8636 SIZE (rust/crates/pyo3-ffi-0.22.6.crate) = 69352 SHA256 (rust/crates/pyo3-macros-0.22.6.crate) = 0fd24d897903a9e6d80b968368a34e1525aeb719d568dba8b3d4bfa5dc67d453 SIZE (rust/crates/pyo3-macros-0.22.6.crate) = 8182 SHA256 (rust/crates/pyo3-macros-backend-0.22.6.crate) = 36c011a03ba1e50152b4b394b479826cad97e7a21eb52df179cd91ac411cbfbe SIZE (rust/crates/pyo3-macros-backend-0.22.6.crate) = 66312 SHA256 (rust/crates/quote-1.0.38.crate) = 0e4dccaaaf89514f546c693ddc140f729f958c247918a13380cccc6078391acc SIZE (rust/crates/quote-1.0.38.crate) = 31252 SHA256 (rust/crates/regex-1.11.1.crate) = b544ef1b4eac5dc2db33ea63606ae9ffcfac26c1416a2806ae0bf5f56b201191 SIZE (rust/crates/regex-1.11.1.crate) = 254170 SHA256 (rust/crates/regex-automata-0.4.9.crate) = 809e8dc61f6de73b46c85f4c96486310fe304c434cfa43669d7b40f711150908 SIZE (rust/crates/regex-automata-0.4.9.crate) = 618525 SHA256 (rust/crates/regex-syntax-0.8.5.crate) = 2b15c43186be67a4fd63bee50d0303afffcef381492ebe2c5d87f324e1b8815c SIZE (rust/crates/regex-syntax-0.8.5.crate) = 357541 SHA256 (rust/crates/rustc-hash-1.1.0.crate) = 08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2 SIZE (rust/crates/rustc-hash-1.1.0.crate) = 9331 SHA256 (rust/crates/serde-1.0.217.crate) = 02fc4265df13d6fa1d00ecff087228cc0a2b5f3c0e87e258d8b94a156e984c70 SIZE (rust/crates/serde-1.0.217.crate) = 79019 SHA256 (rust/crates/serde_derive-1.0.217.crate) = 5a9bf7cf98d04a2b28aead066b7496853d4779c9cc183c440dbac457641e19a0 SIZE (rust/crates/serde_derive-1.0.217.crate) = 57749 SHA256 (rust/crates/syn-2.0.98.crate) = 36147f1a48ae0ec2b5b3bc5b537d267457555a10dc06f3dbc8cb11ba3006d3b1 SIZE (rust/crates/syn-2.0.98.crate) = 297807 SHA256 (rust/crates/target-lexicon-0.12.16.crate) = 61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1 SIZE (rust/crates/target-lexicon-0.12.16.crate) = 26488 SHA256 (rust/crates/unicode-ident-1.0.16.crate) = a210d160f08b701c8721ba1c726c11662f877ea6b7094007e1ca9a1041945034 SIZE (rust/crates/unicode-ident-1.0.16.crate) = 47684 SHA256 (rust/crates/unindent-0.2.3.crate) = c7de7d73e1754487cb58364ee906a499937a0dfabd86bcb980fa99ec8c8fa2ce SIZE (rust/crates/unindent-0.2.3.crate) = 7306

Collapse this list.

Packages (timestamps in pop-ups are UTC):
py39-tiktoken
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest--------
FreeBSD:13:quarterly------0.5.20.5.2
FreeBSD:14:latest--------
FreeBSD:14:quarterly------0.5.20.5.2
FreeBSD:15:latest--n/a-n/a-0.5.2_1-
 

py311-tiktoken
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest0.9.0_10.9.0_1--0.9.0_1---
FreeBSD:13:quarterly0.8.0_20.8.0_2-0.8.0_20.8.0_2---
FreeBSD:14:latest0.9.0_10.9.0_1--0.9.0_1---
FreeBSD:14:quarterly0.8.0_20.8.0_2-0.8.0_20.8.0_2---
FreeBSD:15:latest0.9.0_10.9.0_1n/a0.9.0_1n/a---
 

Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. py311-setuptools>=62.4 : devel/py-setuptools@py311
  2. py311-setuptools-rust>=1.5.2 : devel/py-setuptools-rust@py311
  3. py311-wheel>=0 : devel/py-wheel@py311
  4. rust>=1.85.0 : lang/rust
  5. python3.11 : lang/python311
  6. py311-build>=0 : devel/py-build@py311
  7. py311-installer>=0 : devel/py-installer@py311
Test dependencies:
  1. python3.11 : lang/python311
Runtime dependencies:
  1. py311-regex>=2022.1.18 : textproc/py-regex@py311
  2. py311-requests>=2.26.0 : www/py-requests@py311
  3. python3.11 : lang/python311
This port is required by:
for Run
  1. devel/py-ddtrace
  2. misc/py-aider-chat
  3. misc/py-litellm
  4. misc/py-openai-whisper
Configuration Options:
===> The following configuration options are available for py311-tiktoken-0.9.0_1: BLOBFILE=off: Blob file support ===> Use 'make config' to modify these settings
Options name:
textproc_py-tiktoken
USES:
cargo python:3.9+
FreshPorts was unable to extract/find any pkg message
Master Sites:
Expand this list (2 items)
Collapse this list.
  1. https://files.pythonhosted.org/packages/source/t/tiktoken/
  2. https://pypi.org/packages/source/t/tiktoken/
Collapse this list.

Number of commits found: 22

Commit History - (may be incomplete: for full details, see links to repositories near top of page)
CommitCreditsLog message
0.9.0_1
24 Feb 2025 07:55:52
commit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfb files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.85.0

PR:		284884
0.9.0
19 Feb 2025 11:54:48
commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01 files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.9.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.8.0_3
20 Jan 2025 11:06:50
commit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7f files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.84.0

PR:		283962
0.8.0_2
01 Dec 2024 09:24:18
commit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2f files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.83.0

PR:		283000
0.8.0_1
08 Nov 2024 08:24:20
commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.82.0

PR:		282516
0.8.0
14 Oct 2024 05:25:48
commit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61b files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.8.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.7.0_3
10 Sep 2024 11:00:34
commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.81.0

PR:		281300
0.7.0_2
10 Sep 2024 10:58:07
commit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5ef files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer
Author: Siva Mahadevan		 
*/*: remove STRIP_CMD calls in rust based ports

This is not needed after bc4fedc1fec0d359365c04d43be9e32bf101a50e

PR:		246993
Differential Revision:	https://reviews.freebsd.org/D46503
0.7.0_2
26 Aug 2024 08:08:01
commit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bff files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.80.1

PR:		280490
0.7.0_1
18 Jun 2024 10:59:14
commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.79.0

PR:		279707
0.7.0
19 May 2024 17:15:56
commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3 files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.7.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.6.0_2
13 May 2024 11:03:24
commit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8e files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.78.0

PR:		278834
0.6.0_1
23 Mar 2024 09:41:46
commit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45a files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.77.0

PR:		277786
0.6.0
21 Feb 2024 15:08:04
commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18 files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.6.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.5.2_2
19 Feb 2024 11:59:23
commit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433c files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.76.0

PR:		276920
0.5.2_1
09 Jan 2024 12:15:54
commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.75.0

PR:		276033
0.5.2
14 Dec 2023 05:48:32
commit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6f files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.5.2

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.5.1_2
08 Dec 2023 15:04:09
commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.74.1

PR:		275243
0.5.1_1
24 Oct 2023 10:22:32
commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.73.0

PR:		274499
0.5.1
22 Sep 2023 03:19:31
commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8 files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Update to 0.5.1

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.4.0_1
08 Sep 2023 09:06:24
commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8 files touched by this commit		 Mikael Urankar (mikael) search for other commits by this committer 
lang/rust: Bump revisions after 1.72.0

PR:		273331
0.4.0
21 Aug 2023 17:01:58
commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3 files touched by this commit		 Po-Chuan Hsieh (sunpoet) search for other commits by this committer 
textproc/py-tiktoken: Add py-tiktoken 0.4.0

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Number of commits found: 22
Login
User Login
Create account

Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD

This site
What is FreshPorts?
About the authors
Issues
FAQ
How big is it?
Security Policy
Privacy
Blog
Contact

Search
Enter Keywords:
 
more...

Latest Vulnerabilities
chromiumApr 05
ungoogled-chromiumApr 05
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefoxApr 04
firefox-esrApr 04
firefox-esrApr 04
firefox-esrApr 04
firefox-esrApr 04

28 vulnerabilities affecting 309 ports have been reported in the past 14 days

* - modified, not new

All vulnerabilities

Last processed:
2025-04-05 14:02:13 UTC


Ports
Home
Categories
Deleted ports
Sanity Test Failures
Newsfeeds

Statistics
Graphs
NEW Graphs (Javascript)

Calculated hourly:
Port count 33248
Broken 129
Deprecated 281
Ignore 244
Forbidden 1
Restricted 2
No CDROM 1
Vulnerable 27
Expired 33
Set to expire 238
Interactive 0
new 24 hours 2
new 48 hours3
new 7 days9
new fortnight38
new month92
Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD 		Valid HTML, CSS, and RSS. Copyright © 2000-2025 Dan Langille. All rights reserved.