Veilفارسی

Persian token cost

87
40
2.17×
1.02
0.45
پنجرهٔ6 1متن3 1بیشینهٔ7 1تعداد5 1توکن‌هایی10 1است3 1که2 1یک2 1مدل3 1می‌تواند9 1در2 1یک2 1درخواست7 1در2 1نظر3 1بگیرد.6

This shows how many tokens a Persian text costs and compares it with the same meaning in English. Because Persian is under-represented in tokenizer vocabularies, the same idea usually costs two to four times as many tokens — which means paying several times more and reaching a context limit sooner.

Method and limits: this is an estimate from UTF-8 byte length, not a real tokenizer run. A real merge table has hundreds of thousands of entries and shipping one to the browser would cost megabytes. The ratio it shows is right; the exact number differs a little per model.