Real-time monitoring of LILA Lab's language intelligence pipelines.
$ ./monitor --pipelines --live
Validated evidence that narrative indices work for low-resource languages
Bangla-language economic news narratives show strong negative correlation with CPI and exchange rate movements — the first evidence that narrative indices work for low-resource languages.
Simple TF-IDF + Logistic Regression achieves 91.7% accuracy, outperforming complex deep learning models while requiring no GPU and costing $0.02/article in annotation.
Complete processing of 664,000+ Bangla news articles from the Potrika corpus, creating the largest low-resource language narrative dataset for economic analysis.
Claude and GPT-4o ensemble achieves human-level annotation quality at $0.02–0.03 per article, making large-scale annotation feasible for low-resource languages.
Every language gets its own XENI — proven in Bangla as BENI, ready for yours.
Aggregate millions of native-language articles from local sources, preserving linguistic authenticity from day one.
LLM ensemble (Claude, GPT-4o) annotates narratives across domains. Multi-model classification achieves 91.7% accuracy.
Construct monthly narrative indices validated against macroeconomic indicators. CPI correlation at r = −0.75.
Live charts from the BENI pipeline analysis
Inverse correlation between narrative index and inflation
TF-IDF baseline vs. deep learning models
Consistent data collection across 79 months
Cost per article by annotation method
Language pipeline development progress across all targets
Bangla (বাংলা)
265M speakers
Assamese (অসমীয়া)
15M speakers
Nepali (नेपाली)
25M speakers
Sylheti (ꠍꠤꠟꠐꠤ)
11M speakers
Recent findings, publications, and community announcements
Our validation shows that economic narratives in Bangla news articles correlate strongly with macroeconomic indicators (CPI r = −0.75, FX r = −0.72). This is the first evidence that narrative indices work for low-resource languages.
After optimizing the TF-IDF + Logistic Regression pipeline, we've achieved 91.7% classification accuracy on gold-standard human annotations — without any GPU.
We're seeking native speakers to help build the next XENI pipelines. Each contributor receives co-authorship on the resulting paper. No coding required.
Our systematic review of 20 years of economic narrative indices has been submitted to arXiv. The paper replicates and extends the full ENI methodology literature.
The complete BENI dataset with 664K articles, annotations, and indices is now available on Zenodo with a permanent DOI for reproducible research.
We compare annotation costs and quality between Claude and GPT-4o for Bangla economic relevance labeling. Claude achieves comparable quality at lower cost.
Join our research collective and help build NLP infrastructure for underserved languages.