How AI follows Quran recitation word by word (and why it runs on your phone)
You open the mushaf, start reciting, and the word you are on lights up. Turn the page in your head and the screen turns with you. Start in the middle of Surah Al-Kahf with the app showing Al-Fatiha, and a few words later it has found you. That is what a Quran recitation tracker does. This post explains how ours works, in recite.nabilmh.com and in the Naml tracker code, and why none of it needs a server.
Two jobs: hear, then place
A tracker does two separate things.
- Hear. A speech model turns audio into symbols: letters, or the phonetic units of tajweed. It does not know where in the Quran you are; it only writes down what it heard, a fraction of a second after you said it.
- Place. A tracker compares that stream of symbols with the text of the Quran and decides which word you are on. This is where the Quran helps: unlike general dictation, the text is known in advance. The question is never "what did they say?" but "where in these 6,236 ayahs are they?".
Keeping the two apart is what makes the system small. The model does not have to be a perfect transcriber, because the tracker only needs enough evidence to tell one place from another.
Hearing: a streaming model
Our models (Naml Tajweed, Naml Letters and Naml Letters Plus) are streaming FastConformer models. Every 0.32 seconds they take the newest slice of audio, look 0.16 seconds ahead, and emit what they heard. They remember the last 5.6 seconds of context in a cache, so each step only computes the new part.
You can watch this happen on the model demo: three rows of output growing as the recording plays, with the time each step took. On one server CPU core a step takes about 15 ms out of its 320 ms budget. On a budget Android phone it is about 18 ms, which is roughly 6% of real time. That headroom is the reason it can run in a browser tab while the phone does everything else.
Naml Tajweed, the default, does not output plain letters. It outputs 77 tajweed phoneme units: a madd of 2, 4 or 6 counts, a doubled consonant for shadda, the nasal sounds of ghunna, ikhfa and iqlab. Those units match how the Quran is actually recited, which helps the tracker and makes tajweed feedback possible later (how the units work).
Placing: alignment, search and page turns
The tracker keeps a window of recent symbols and aligns it against the expected text around your current position. If the alignment is strong, the cursor moves forward word by word. If it weakens (you paused, coughed, repeated an ayah, or jumped), it widens the search. In Search mode it looks through the whole Quran from your first words.
A few behaviours matter in practice:
- Repeats. Reciters go back to restart an ayah all the time. The tracker has to follow the repeat without deciding you skipped ahead.
- Mutashabihat. Some passages are nearly identical in different surahs. A tracker that commits too early will jump to the wrong one (more on this).
- Page turns. When you reach the last word of a page, the next page should already be there.
- The leading cursor. The model's confirmation always arrives a little after the sound. The app carries your pace forward so the highlight does not lag behind your voice.
The tracker demo replays all of this from a phone recording of Al-Mulk: you can see the cursor lock on in the first seconds and follow to the end of the page.
Why on the phone, not in the cloud
Sending recitation to a server works, and many apps do it. Running it on the device is better in four ways.
- Privacy. A child practising hifz, a person reciting alone at night: their voice never leaves the device. There is nothing to store, leak or subpoena.
- Cost. A cloud speech API charges per minute. A Quran app with many daily users pays for every one of those minutes, every day. On-device, the marginal cost is zero.
- Latency. No round trip. The highlight follows your voice at the speed of the phone's CPU, not the speed of the network.
- Offline. After the first download it works in a masjid basement or on a plane.
The cost of on-device is size: the model has to be small. Ours store each weight in 2 bits (−1, 0 or +1 times a scale), so a model is about 17 MB (why ternary works). They run on a hand-written WebAssembly SIMD core rather than a general runtime (WebAssembly vs WebGPU).
What it still gets wrong
Honesty about limits matters more than a demo.
- Hard audio. Heavy reverberation in a big masjid, rain on the window, a whisper: the model hears less, and the tracker can lose its place for a few seconds before it finds you again.
- Very fast hadr. At the fastest recitation speeds the model drops more units.
- Long, similar passages. Mutashabihat can still pull the cursor to the wrong surah for a moment.
- Timing. On 31 reviewed clips, Naml Letters puts both the start and the end of an ayah within 250 ms for 51% of ayahs and within 500 ms for 68%. That is ahead of the wav2vec2 aligners we compared (43% and 62% with Viterbi alignment), but it is not perfect (the benchmark).
Try it
Open recite.nabilmh.com, press Start and recite from anywhere. If you are building an app, the models are free under CC BY 4.0 and the tracker code that runs the site is for sale.