Interactive Tokenization Visualizer

Compare BPE and WordPiece algorithms

⌨️ Keyboard: ← → to navigate | ` to toggle algorithm

🎓 How the Algorithm Works

Input Text

or enter your own text below:

Tokenized Text

Legend: a Single character ab 2 chars abc 3 chars abcd 4+ chars new Just merged
⏭️ Next Step:
✓ Last Step:
Step 0 of 0

Vocabulary

Size: 0 tokens

Next Pairs Available

Test Current Vocabulary

Enter new text below to see how the current BPE vocabulary tokenizes it: