keywords:

BPE

found 13 packages in 2ms

gpt-tokenizer730.1kMIT3.0.1

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

niieani17 days agoBPE, encoder, decoder, tokenizer

gptoken5.2kMIT0.1.0

Javascript BPE Encoder Decoder for GPT-2 / GPT-3. The "gpt-3-encoder" module provides functions for encoding and decoding text using the Byte Pair Encoding (BPE) algorithm. It can be used to process text data for input into machine learning models, or to

syonfoxover 2 years agoJavaScript, BPE, Encoder, Decoder

npmGitHubHomepage

@syonfox/gpt-3-encoder105MIT1.4.0-rc5

Javascript BPE Encoder Decoder for GPT-2 / GPT-3. The "gpt-3-encoder" module provides functions for encoding and decoding text using the Byte Pair Encoding (BPE) algorithm. It can be used to process text data for input into machine learning models, or to

latitudegamesover 2 years agoJavaScript, BPE, Encoder, Decoder

n8n-nodes-gpt-tokenizer117MIT0.1.1

n8n node for working with BPE Tokens with OpenAI's GPT models in mind.

geckseabout 2 years agon8n-community-node-package, n8n, nodemation, n8n-node

@meistrari/gpt-tokenizer85MIT0.3.2

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 / Claude Instant / Claude 2

meistrari7 months agoBPE, encoder, decoder, tokenizer

‹
1
2
3
›

Packages

Lightweight debugging utility for Node.js and the browser

debug, log, debugger

Lodash modular utilities.

modules, stdlib, util

React is a JavaScript library for building user interfaces.

A querystring parser that supports nesting and arrays, with a depth limit

querystring, qs, query, url

Higher-order functions and common patterns for asynchronous code

async, callback, module, utility

babel babel-core

Babel compiler core.

6to5, babel, classes, const