Building a Quran app with offline speech recognition: privacy, cost and latency
If you build a Quran app, sooner or later users ask for it to listen: follow along as they recite, check their hifz, catch a mistake. The usual way is a cloud speech API. This post is about the other way, running recognition on the user's device, and what to weigh when you choose.
Three questions to ask first
Where does the audio go? Recitation is personal. Many of your users are children. With a cloud API, every session is uploaded, processed and, depending on the provider, kept. You then need a privacy policy that says so, consent flows, and a plan for data requests. On the device, the audio never leaves the phone, and the policy is one sentence.
What does a minute cost? Cloud speech is priced per minute of audio. A Quran app's usage pattern (daily practice, long sessions, Ramadan peaks) multiplies that. TODO: compare a typical per-minute price for your expected usage before deciding. On-device, the marginal cost of a minute is zero; you pay once for the model download, which you can serve from any CDN.
How fast does it need to feel? A highlight that follows the voice has to be quick. A server adds a network round trip on every update, and mobile networks vary. On-device, the delay is the model's look-ahead (0.16 s for our models) plus compute (about 15-18 ms per 0.32 s step on current phones), with no network in the loop.
What on-device costs you
- Download size. Our models are about 17 MB of weights each plus a 6 MB front end, downloaded once and cached (how they got that small).
- Device range. Old browsers without WebAssembly SIMD fall back to a much slower path. Current Chrome, Edge, Firefox and Safari all have SIMD.
- Accuracy limits. A small domain model is not a general dictation engine. It is trained on recitation; it will not transcribe a lecture well. For Quran tracking that is the point.
- Your engineering time. A model gives you letters or phonemes every 0.32 s. Turning that into "the user is on word 7 of 67:3, turn the page" is the larger part of the work.
The pieces of a recitation feature
- Audio capture: microphone at 16 kHz, in an AudioWorklet, resampled if the browser ignores the requested rate.
- The model: a streaming step every 0.32 s, in a Web Worker so the page stays smooth.
- The tracker: aligns the model's output to the Quran text around the current position, searches the whole Quran when it loses you, handles repeats and similar passages (how).
- The page: the mushaf with the current word highlighted, page turns, and the views users ask for (a large single-ayah view, a prayer view, a hifz test).
- Optional feedback: tajweed analysis per word.
Three ways to get there
| Option | You get | Cost |
|---|---|---|
| Free model | The three Naml models, the wasm core, unit lists, a JavaScript runner. You build steps 1, 3, 4. | Free, CC BY 4.0 (credit us and NVIDIA) |
| Tracker code | The full source of recite.nabilmh.com: all five pieces except tajweed. | $500 one-time |
| Masjid kit | The same source set up as a Tarawih screen for a masjid, with a setup guide. | $500 one-time |
| Tajweed correction | The tracker code plus the per-word tajweed analyser, Mistake History screen and calibration script. | coming soon |
If you want it built into your app for you, the fourth option is an integration project.
Questions we get from developers
Native apps? The engine is TypeScript and WebAssembly. It runs in a WebView, or the wasm core can be hosted by any wasm runtime. TODO: a native iOS / Android sample.
Can I use my own Quran text or mushaf? The tracker works on the Tanzil text and word indices; the mushaf view uses a page layout. Other layouts need a mapping from their words to those indices.
Which qira'a? Hafs an Asim. Other readings would need training data and a different phonetizer.