MinbarLive — Technical Architecture

MinbarLive

Technical Architecture & Implementation

Tehnička Arhitektura i Implementacija

Building Real-Time
Multilingual Sermon
Delivery Izgradnja Real-Time
Višejezičnog Sistema
Isporuke Hutbe

A deep-dive into the architecture, AI pipeline, and infrastructure powering live khutbah captions for mosques worldwide.

Detaljan pregled arhitekture, AI pipeline-a i infrastrukture koja pokreće live titlove hutbe za džamije širom svijeta.

VersionVerzija2026-04

AudiencePublikaDevelopers / IT ManagersDeveloperi / IT Menadžeri

URLminbarlive.com

System Overview

Pregled Sistema

What MinbarLive Is Technically

Šta je MinbarLive Tehnički

A multi-module web platform for live sermon captions with real-time AI transcription, multilingual translation, video studio, podcast companion, AI sermon preparation, and persistent archive.

Multi-modularna web platforma za live titlove hutbe s real-time AI transkripcijom, višejezičnim prijevodom, video studijem, podcast pratiocen, AI pripremom hutbi i trajnom arhivom.

Public Surface

Javna Površina

Global directory

Globalni direktorij

Live reader

Live čitač

Core Technologies

Ključne Tehnologije

Frontend

Next.js 15 React 19 Tailwind CSS 4 Lucide Icons

Backend

FastAPI (Python) Socket.IO WebSockets

Data & Auth

Podaci i Auth

Supabase (PostgreSQL) MongoDB (sessions/cache) Socket.IO (real-time)

AI / ML

Deepgram STT OpenAI GPT-4o-mini OpenAI Whisper (Studio) Dynamic prompts Keyword boosting

Session & State

Sesija i Stanje

Redis MongoDB TTL sessions ZSET rate limiting Job tracking

Infrastructure

Infrastruktura

FFmpeg / ffprobe SendGrid (email) Correlation IDs Audit log

Architecture

Arhitektura

System Architecture Diagram

Dijagram Sistemske Arhitekture

Client Layer — Audience Web/Mobile

Klijentski Sloj — Audience Web/Mobile

Public API calls

Public API pozivi

Socket.IO real-time delivery

Socket.IO real-time isporuka

HTTP polling fallback

↑ ↓

Admin Web/Mobile

Admin API (auth, stream, transcript, billing)

Admin API (auth, stream, transkript, naplata)

Ingest Gateway (browser/phone/mixer)

Ingest Gateway (browser/telefon/mikser)

Socket.IO (audio chunks)

Socket.IO (audio chunk-ovi)

↑ ↓

Backend Services (FastAPI)

Backend Servisi (FastAPI)

Audio pipeline (STT orchestration)

Audio pipeline (STT orkestracija)

Translation service

Servis prijevoda

Transcript service

Servis transkripta

Billing engine

Studio pipeline (FFmpeg+Whisper)

Quality analyzer

AI Providers

AI Provajderi

Deepgram (STT — 20 lang)

OpenAI GPT-4o-mini (translation + transcript)

OpenAI Whisper (Studio STT)

↑ ↓

Core Data Layer

Centralni Sloj Podataka

Supabase PostgreSQL

Supabase Auth

MongoDB

Local file storage (config)

Lokalni fajlovi (config)

Session & Runtime State

Sesija i Runtime Stanje

Redis (sessions)

Redis (rate limit)

Redis (audio registry)

Redis (job tracking)

Core Pipeline

Centralni Pipeline

Audio Processing Flow

1

Audio Capture

Snimanje Zvuka

Browser mic (AudioWorklet with ScriptProcessorNode fallback), phone upload, or external mixer. Admin selects ingest mode at session start.

Browser mikrofon (AudioWorklet sa ScriptProcessorNode fallbackom), upload s telefona, ili eksterni mikser. Admin bira mode pri startu sesije.

2

Chunked Streaming

Audio chunks sent via Socket.IO to backend. Smart buffering + segmentation for smooth subtitle display.

Audio chunk-ovi šalju se putem Socket.IO na backend. Smart buffering + segmentacija za glatki prikaz titlova.

3

Deepgram STT

Real-time speech-to-text via Deepgram WebSocket streaming. 20 source languages + auto-detect. Islamic vocabulary keyword boosting for BS/HR/SR.

Real-time speech-to-text putem Deepgram WebSocket streaminga. 20 izvornih jezika + auto-detekcija. Keyword boosting islamskog rječnika za BS/HR/SR.

4

AI Translation

AI Prijevod

OpenAI GPT-4o-mini translates via two-step pipeline: normalize mixed content to Croatian, then translate to target language. On-demand for 135+ languages.

OpenAI GPT-4o-mini prevodi putem dvostepenog pipeline-a: normalizacija miješanog sadržaja na hrvatski, zatim prijevod na ciljni jezik. On-demand za 135+ jezika.

5

Real-time Delivery

Real-time Isporuka

Translated segments pushed via Socket.IO broadcast to all connected listeners. Pre-computed languages stored in Supabase, on-demand in MongoDB. HTTP polling as fallback.

Prevedeni segmenti šalju se putem Socket.IO broadcasta svim slušaocima. Pre-computed jezici u Supabase, on-demand u MongoDB. HTTP polling kao fallback.

6

Archive & Transcript

Arhiva i Transkript

Session ends → transcript generation triggered (background thread, Redis job tracking). Export available per language.

Sesija završi → generacija transkripta pokreće se (background thread, Redis job tracking). Export dostupan po jeziku.

Access Control

Kontrola Pristupa

User Role Model

Model Korisničkih Uloga

Owner

Single account. Full platform access. Approvals, rejections, user governance, cross-org oversight.

Jedan account. Puni pristup platformi. Odobravanja, odbijanja, upravljanje korisnicima.

Full access Puni pristup

Super Admin

Scoped to managed_organization_ids. Manages multiple orgs, billing, analytics, admins. No owner-only actions.

Scoped na managed_organization_ids. Upravlja više org, billing, analitika, admini. Bez owner-only akcija.

Scoped multi-org Scoped multi-org

Admin

Single organization. Controls session, capture, archive, billing view for their mosque.

Jedna organizacija. Kontroliše sesiju, snimanje, arhivu, billing pregled za svoju džamiju.

Single org Jedna org

Public

Korisnik

No authentication needed. QR scan → join live session → read transcripts. Anonymous.

Bez autentifikacije. QR scan → join live sesiju → čitanje transkripta. Anonimno.

No auth Bez auth

Auth Implementation

Implementacija Auth-a

• Custom auth (email/password with bcrypt hashing) Custom auth (email/lozinka s bcrypt hashiranjem)

• Session tokens stored in MongoDB/Redis with TTL auto-expiry Session token-i u MongoDB/Redisu s TTL auto-expiryjem

• Per-email brute force lockout (5 attempts) Per-email brute force zaključavanje (5 pokušaja)

• Rate limiting via Redis ZSET sliding window Rate limiting putem Redis ZSET sliding window-a

• Registration requires privacy + terms consent Registracija zahtijeva pristanak na privacy + terms

Organization Hierarchy

Hijerarhija Organizacija

Master org Master org Child orgs Child org managed_organization_ids

Codebase

Key Modules & Files

Ključni Moduli i Fajlovi

Backend (FastAPI + Python — 56 routers, 455 endpoints)

Backend (FastAPI + Python — 56 routera, 455 endpointa)

›server.py — app assembly, Socket.IO handlers, lifespan (402 lines)sklapanje app-a, Socket.IO handleri, lifespan (402 linije)

›audio_pipeline.py — STT orchestration, Deepgram WebSocket, mixed Arabic/Latin detectionSTT orkestracija, Deepgram WebSocket, miješana Arabic/Latin detekcija

›pipeline_persistence.py — Two-step translation, BCS heuristic, on-demand cacheDvostepeni prijevod, BCS heuristika, on-demand cache

›studio/ — 10 modules — pipeline, queue, audio, transcription, translation, subtitles10 modula — pipeline, queue, audio, transkripcija, prijevod, titlovi

›routers/admin_khutbah_kb.py — HutbaAsistent AI (generate, refine, templates, versions)HutbaAsistent AI (generiranje, doraba, predlošci, verzije)

›routers/admin_podcasts.py — Podcast Companion (CRUD, live stream, QR)Podcast Companion (CRUD, live stream, QR)

›routers/public_stream_routes.py — On-demand translation + stale detection + typing UXOn-demand prijevod + stale detekcija + typing UX

›costing/ — 6 billing modules (plans, invoices, PDF, email dispatch)6 billing modula (planovi, fakture, PDF, email dispatch)

›271 Python modules, ~111,000 lines (+ 295 test files / ~59,000 lines)

Frontend (Next.js 15 + React 19)

›src/app/(admin)/ — 60+ admin pages (HutbaLive, Studio, Podcast, HutbaAsistent, Billing)60+ admin stranica (HutbaLive, Studio, Podcast, HutbaAsistent, Billing)

›src/app/join/ — Live session listener (Socket.IO, 135+ lang, typing animation)Live session slušalac (Socket.IO, 135+ jezika, typing animacija)

›src/lib/api.js — API client + auth session managementAPI klijent + auth session upravljanje

›src/lib/stream-realtime.js — Socket.IO real-time client + translation handlersSocket.IO real-time klijent + translation handleri

›src/lib/browser-audio-capture.js — AudioWorklet capture + ScriptProcessor fallbackAudioWorklet capture + ScriptProcessor fallback

›src/lib/translations/ — 9 languages × 2,900+ i18n keys (~26,000 strings)9 jezika × 2.900+ i18n ključeva (~26.000 stringova)

›src/components/ — LanguagePicker, OEmbedPreview, Sidebar, Header, UI kitLanguagePicker, OEmbedPreview, Sidebar, Header, UI kit

›public/audio-capture-processor.js — AudioWorklet processor (separate audio thread)AudioWorklet procesor (odvojeni audio thread)

›420 source files, ~90,000 lines

Security & Observability

Sigurnost i Observabilnost

Security & Monitoring

Sigurnost i Praćenje

Security Controls

Sigurnosne Kontrole

✓Redis session store with TTL auto-expiryRedis session store s TTL auto-expiryjem

✓Rate limiting via ZSET sliding window (per IP/user)Rate limiting putem ZSET sliding window-a

✓Privileged action audit log (JSONL)Audit log privilegiranih akcija (JSONL)

✓Consent checkboxes on registrationConsent checkboxovi pri registraciji

✓Supabase Row Level Security (RLS)Supabase Row Level Security (RLS)

⚠Audit log not yet cluster-safe (local JSONL)Audit log nije cluster-safe (lokalni JSONL)

⚠No formal key rotation / secrets managementBez formalnog key rotation / secrets management-a

Observability

Observabilnost

✓RequestCorrelationMiddleware (X-Correlation-ID)

✓Structured request loggingStrukturirani request logging

✓Deep health endpoint (degraded/healthy status)Deep health endpoint (degraded/healthy status)

✓Operator notification ring bufferOperator notification ring buffer

✓Capture health strip (admin UI)Capture health strip (admin UI)

✓14 smoke tests (backend/tests/smoke_test.py)

✓Event timeline in admin control panelEvent timeline u admin control panelu

Billing / Cost Tracking

Billing / Praćenje Troškova

6-module billing engine: fixed costs, usage events (Deepgram + OpenAI), org allocation (equal/custom), daily breakdowns, monthly reports, markup separation.

6-modularni billing engine: fiksni troškovi, usage eventi (Deepgram + OpenAI), org alokacija (jednaka/custom), dnevni pregledi, mjesečni izvještaji, markup odvajanje.

Studio Mode

Video Processing Pipeline

Beyond live sessions — Studio Mode processes pre-recorded video files through the same AI pipeline, producing synchronized subtitles and styled exports.

Izvan live sesija — Studio Mode obrađuje unaprijed snimljene video fajlove kroz isti AI pipeline, proizvodeći sinkronizirane titlove i stilizirane exporte.

1

Video Upload

Upload Videa

Admin uploads video file via Studio UI. Chunked upload bypasses proxy limits. Billing plan gating ensures only authorized orgs access Studio.

Admin uploada video putem Studio UI-a. Chunked upload zaobilazi proxy limite. Billing plan gating osigurava pristup samo ovlaštenim org.

2

FFmpeg Audio Extraction

FFmpeg Ekstrakcija Zvuka

Server-side FFmpeg extracts audio track from video. ffprobe validates format and duration. Supports MP4, MKV, WebM, MOV.

Server-side FFmpeg ekstrahuje audio iz videa. ffprobe validira format i trajanje. Podržava MP4, MKV, WebM, MOV.

3

OpenAI Whisper STT

Extracted audio sent to OpenAI Whisper for high-accuracy transcription with word-level timestamps. Segments stored with start_ms/end_ms precision.

Ekstrahirani audio šalje se na OpenAI Whisper za transkripciju visoke tačnosti s timestamps na nivou riječi. Segmenti se čuvaju s start_ms/end_ms preciznosti.

4

AI Translation

AI Prijevod

Transcribed segments translated via OpenAI GPT. Same Islamic-terminology-tuned prompts as live mode. Multi-language output per segment.

Transkribovani segmenti prevode se putem OpenAI GPT. Isti promptovi prilagođeni islamskoj terminologiji kao u live modu. Višejezični output po segmentu.

5

Video Preview with Subtitles

Video Preview s Titlovima

Blob URL-based video player with real-time subtitle overlay. Authenticated streaming without exposing tokens. Custom React component isolates HTML5 video from React Native View constraints.

Video player baziran na Blob URL-u s real-time subtitle overlay-om. Autentificirani streaming bez izlaganja tokena. Custom React komponenta izolira HTML5 video od React Native View ograničenja.

6

Styled HTML Export

Stilizirani HTML Export

4 export formats: plain transcript, styled transcript (Source Serif 4 font), styled translation, bilingual side-by-side. Print-ready CSS with responsive design.

4 export formata: obični transkript, stilizirani transkript (Source Serif 4 font), stilizirana prijevod, dvojezični side-by-side. Print-ready CSS s responsive dizajnom.

Quality Assurance

Osiguranje Kvalitete

Continuous Quality Loop

Kontinuirani Quality Loop

An automated feedback system that analyzes every session, scores quality, identifies issues, and generates actionable improvement suggestions.

Automatizirani feedback sistem koji analizira svaku sesiju, boduje kvalitetu, identificira probleme i generira djelotvorne prijedloge za poboljšanje.

Quality Dashboard

•Stats row: Sessions Analyzed, Average Score, Pending Suggestions, TrendStats row: Analizirane Sesije, Prosječni Score, Pending Sugestije, Trend

•Reports tab: expandable cards with quality score ring, AI summary, issue breakdownReports tab: expandable kartice s quality score ring-om, AI sažetkom, pregledom problema

•Suggestions tab: filter by status (all/pending/applied/dismissed), apply/dismiss/reopen actionsSuggestions tab: filter po statusu (sve/pending/applied/dismissed), apply/dismiss/reopen akcije

•Add Suggestion tab: manual submission with priority, type, and technical detailAdd Suggestion tab: manualno podnošenje s prioritetom, tipom i tehničkim detaljima

API Endpoints

API Endpointi

›GET /api/admin/quality/report — fetch quality metricsdohvati quality metrike

›POST /api/admin/quality/suggest — submit manual suggestionspodnesi manualne sugestije

›PATCH /api/admin/quality/suggestions/:id — apply/dismiss/reopenapply/dismiss/reopen

Quality Feedback Loop

Session Ends

Sesija Završava

↓

AI Analyzes Quality

AI Analizira Kvalitetu

↓

Score + Issues Report

Score + Izvještaj Problema

↓

Actionable Suggestions

Djelotvorne Sugestije

↓

Apply → Improve

Primijeni → Poboljšaj

Technical Implementation

Tehnička Implementacija

✓quality_analyzer.py — rule-based + AI analysisquality_analyzer.py — rule-based + AI analiza

✓MongoDB collections: quality_reports, quality_suggestionsMongoDB kolekcije: quality_reports, quality_suggestions

✓Owner/Super Admin access only (role-gated)Samo Owner/Super Admin pristup (role-gated)

Current Status

Trenutni Status

Implementation Progress

Napredak Implementacije

Phase 1 — Stabilization

Env-driven config, no hardcoded fallbacksEnv-driven config, bez hardcoded fallbacksDONEGOTOVO

Remove silent failures, surface errorsUklanjanje tihih grešakaDONEGOTOVO

Smoke/integration tests (14 tests)Smoke/integration testovi (14)DONEGOTOVO

Canonical documentationKanonska dokumentacijaDONEGOTOVO

Phase 2 — Optimization

admin.tsx 2776→1053 lines (−62%)DONEGOTOVO

audio_pipeline.py 1253→513 (−59%)DONEGOTOVO

Arabic→Croatian STT transition optimizationOptimizacija AR→HR STT tranzicijeDONEGOTOVO

Translation caching (Redis)Caching prijevoda (Redis)DONEGOTOVO

Phase 3 — Scaling

Redis for sessions/rate limiting/registryRedis za sesije/rate limiting/registryDONEGOTOVO

Durable job queue for transcriptsDurable job queue za transkriptePARTIALDJELIMIČNO

File-backed config → DB migrationMigracija config fajlova → DBPLANNEDPLANIRANO

Phase 4 — Enterprise

Billing/cost tracking foundationOsnova billing/cost trackingaDONEGOTOVO

Studio Mode — video processing pipelineStudio Mode — video processing pipelineDONEGOTOVO

Quality Dashboard — feedback loop UIQuality Dashboard — feedback loop UIDONEGOTOVO

Styled HTML transcript exportsStilizirani HTML export transkripataDONEGOTOVO

White-label + custom domainsWhite-label + custom domeneDONEGOTOVO

AudioWorklet capture (replaces deprecated API)AudioWorklet snimanjeDONEGOTOVO

Technical Roadmap

Tehnički Roadmap

Next Workstreams

Sljedeći Radni Tokovi

HutbaLive Core DONE

HutbaLive Core GOTOVO

Live audio capture, Deepgram STT, OpenAI translation, Socket.IO delivery, QR audience access.

Live audio capture, Deepgram STT, OpenAI prijevod, Socket.IO isporuka, QR pristup publike.

Studio + Podcast + HutbaAsistent DONE

Studio + Podcast + HutbaAsistent GOTOVO

Video processing pipeline, live podcast companion, AI sermon preparation with templates and knowledge base.

Video obrada pipeline, live podcast companion, AI priprema hutbi s predlošcima i bazom znanja.

Two-Step Translation Pipeline DONE

Dvostepeni Translation Pipeline GOTOVO

On-demand 135+ languages, stale cache detection, typing animation UX, BCS heuristic.

On-demand 135+ jezika, detekcija stalog cachea, typing animacija UX, BCS heuristika.

Architecture Decomposition DONE

Dekompozicija Arhitekture GOTOVO

56 router files, modular studio, god-file decomposition (−88%), security audit, i18n (9 lang × 2,900+ keys), AudioWorklet.

56 router fajlova, modularni studio, dekompozicija god-fajlova (−88%), sigurnosni audit, i18n (9 jezika × 2.900+ ključeva), AudioWorklet.

Payments & Billing DONE

Plaćanje i Naplata GOTOVO

Stripe + Revolut live. Self-serve subscriptions, usage-based billing, invoices/PDF, EU VAT, coupons, trial period.

Stripe + Revolut uživo. Self-serve pretplate, usage-based naplata, fakture/PDF, EU PDV, kuponi, probni period.

NOW — First Paying Clients Live

SADA — Prvi Plaćajući Klijenti Uživo

First paying mosques onboarded and live every Friday. Scaling the reference base, collecting weekly uptime & accuracy metrics, imam testimonials.

Prve plaćajuće džamije onboardane i uživo svaki petak. Širenje referentne baze, prikupljanje sedmičnih uptime i accuracy metrika, svjedočenja imama.

White-Label Platform DONE

White-Label Platforma GOTOVO

Custom domains, brand settings, DNS verification, multi-tenant branding — live for institutional clients.

Custom domene, brand postavke, DNS verifikacija, multi-tenant branding — uživo za institucionalne klijente.

Q4 2026 — AI Enhancement Layer

Q4 2026 — AI Prošireni Sloj

Speaker identification, cross-session search, auto-tagging, content recommendations.

Identifikacija govornika, pretraga između sesija, auto-tagiranje, preporuke sadržaja.

CTO Assessment: "The product has expanded from a live translation tool into a complete imam workflow platform. ~200,000 lines of core product code across 691 files, 7 modules, 455 API endpoints, 295 test files. Payments (Stripe + Revolut) and white-label are live, and the first paying clients are onboarded — the focus now is scaling repeatable weekly usage." — CTO Brief, June 2026

CTO Procjena: "Proizvod se proširio od alata za live prijevod u kompletnu platformu za workflow imama. ~200.000 linija osnovnog koda u 691 fajlu, 7 modula, 455 API endpointa, 295 test fajlova. Plaćanje (Stripe + Revolut) i white-label su uživo, a prvi plaćajući klijenti su onboardani — fokus je sada na skaliranju ponavljajuće sedmične upotrebe." — CTO Brief, juni 2026