How Accurate Is the Typing Engine?
Measured
When you type vanakkam and press space, the editor has to guess which Tamil word you meant. This page explains how we test that guess, what the numbers are, and where it still goes wrong.
Short answer
On 6,833 test words typed in English letters by native Tamil speakers, words the engine never learned from, the first suggestion is the intended Tamil word 76.2% of the time, and the right word is in the five suggestions shown 84.7% of the time. The test runs automatically on every change to the code.
What We Test
The test data is the Tamil part of the Dakshina dataset (v1.0, Google Research, 2020): Tamil words paired with the ways native speakers wrote them in English letters. A word can have several romanizations, and one romanization can fit several words, so a suggestion counts as right if it matches any Tamil word attested for that input.
Dakshina splits its word list into three parts by Tamil word, so a test word never appears in the training part:
| Set | Size | Used for |
|---|---|---|
| Train | 68,218 pairs | Learning how English letters map to Tamil sounds |
| Dev | 6,806 inputs | The only set used to tune the engine |
| Test | 6,833 inputs | Reported below, never tuned on |
| Running text (test) | 45,831 words | Whole sentences, so common words count more often |
What the Numbers Mean
- Top-1: the first suggestion, the one you get by pressing space, is right.
- Top-5: the right word is somewhere in the five suggestions the editor shows, so you can tap it.
Results
| Test | Top-1 | Top-5 |
|---|---|---|
| Dakshina test words | 76.2% | 84.7% |
| Running text | 77.8% | 84.4% |
| Before the word list was added (test words) | 36.5% | 55.6% |
| Indian names (Aksharantar AK-NEI) | 26.4% | 47.2% |
| Foreign names (Aksharantar AK-NEF) | 23.1% | 35.8% |
The figures in the first two rows use the full 200,000-word list, which loads in the background a moment after the page opens. Until it arrives, the engine works from a smaller built-in list and scores about 40% top-1 on the same test. The name rows come from AI4Bharat’s Aksharantar test sets and are there to show the weak spot honestly.
Checked on Every Change
A frozen sample of 2,000 test words is part of the project’s automated tests. Every change to the code runs it, and the change fails if top-1 drops below 75.5% or top-5 below 84.5%. So the engine cannot quietly get worse.
Where It Still Goes Wrong
- English words and foreign names that are not in its loanword list:
overgives அவர் when you meant ஓவர், andcubagives சுப instead of கியூபா. - Short and long vowels when both readings are real words:
katregives கற்று when you meant காற்றே. The engine looks at one word at a time, not the sentence. - Two real words with one spelling:
puliis honestly both புலி (tiger) and புளி (tamarind). That is why every word has a suggestion list. - Spoken Tamil such as போறேன் relies on a smaller hand-made list, because the word frequencies come from written Tamil.
When you pick a different suggestion, the editor remembers it on your device and puts it first next time. If a common word keeps coming out wrong, tell us what you typed and what you expected.
Data and Credits
The word list, spelling statistics and test sample contain data adapted from the Dakshina dataset v1.0 by Google Research (Brian Roark, Lawrence Wolf-Sonkin, Christo Kirov, Sabrina J. Mielke, Cibu Johny, Isin Demirsahin and Keith Hall, “Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset”, LREC 2020), used under CC BY-SA 4.0. Our adapted data files are shared under the same licence. Aksharantar by AI4Bharat was used only as an extra test; nothing from it ships on the site.
Questions People Ask
Does the reported typing accuracy mean my whole document is correct?
No. The results measure whether an intended Tamil word appears first or among the suggestions for an English-letter input. They do not certify a document’s grammar, meaning or factual accuracy. Review the final text, especially names and ambiguous spellings. For spelling checks, use the separate Tamil spell checker and read its limits.
What is the difference between top-1 and top-5 accuracy?
Top-1 counts inputs where the first suggestion matches an accepted Tamil spelling; top-5 counts inputs where a match appears anywhere in the five suggestions. Choosing another suggestion can therefore correct some first-choice errors. These are measurements on a fixed test set, so neither figure is a guarantee about your next word.
Why are names and spoken Tamil less reliable than ordinary words?
The engine ranks spellings using word-level rules and a dictionary drawn largely from written Tamil. Unfamiliar names and spoken forms may have less dictionary support, while one English spelling can fit several Tamil words. Check a person’s preferred spelling directly. The separate name-test results above show why the ordinary-word score should not be applied to names.
How can developers check which data the accuracy test uses?
The measurement uses held-out Tamil romanization data from Google Research’s Dakshina dataset, with a frozen sample used in the project’s regression tests. Consult the dataset documentation for its splits and collection method. The table above separates word tests, running text and additional name tests because they measure different input distributions.
Sources
Every fact on this page was checked against these official pages on . Products change often; if something here is out of date, tell us.