Interactive Tokenization Visualizer

Compare BPE and WordPiece algorithms

BPE (Frequency-based) WordPiece (Likelihood-based)

⌨️ Keyboard: ← → to navigate | ` to toggle algorithm

Input Text

Choose Example: or enter your own text below:

Step 0 of 0

Tokenized Text

This Step

Vocabulary

Size: 0 tokens

Next Pairs Available

Test Current Vocabulary

Enter new text below to see how the current BPE vocabulary tokenizes it: