How to use
- Paste a document or open a text / Markdown file.
- Set the chunk size and overlap, and choose whether size is measured in tokens (o200k_base) or characters.
- Inspect each chunk — overlapping text is highlighted — then export the chunks as JSON or JSONL for your embedding pipeline.
FAQ
What chunk size should I use?
A common starting point is 300–800 tokens with 10–20% overlap. Smaller chunks give more precise retrieval; larger ones keep more context. Test with your own questions and documents.
How does recursive splitting work?
Text is split at the strongest boundary first — blank lines (paragraphs) — and only pieces that are still too long are split further by lines, sentences, clauses and words. Small pieces are then merged back up to the size limit, so chunks rarely cut through a sentence.