The Urbanixm Knowledge Discovery Engine is an AI powered content processing pipeline for gathering web content about urbanism, extracting information about urban policy and infrastructure interventions, and making that information easier to find. It pays particular attention to the sustainability impact of urban interventions.
Context
Urbanixm tells stories of sustainable urban development through analysis of urban data and by collecting case studies about the impact of urban policy and infrastructure interventions.
My role and contribution
Software Engineer and Applied Data Scientist. I developed the application end-to end.
How it works
The engine collects content from sources such as local authority websites, news sites, urbanism blogs, and research reports. An information-curation pipeline extracts information about urban policy, infrastructure implementation, intervention impacts, and statistical characteristics of places. The resulting knowledge is stored in an urban knowledge repository.
The system includes:
- Web crawler: Finds web pages that may be relevant to urbanism, including scientific articles, project reports, and press releases.
- Knowledge curator: Uses classifiers to identify relevant and quotable pages, urbanism topics, and places. It also annotates smaller text passages, or knowledge snippets, with topics, places, and narration types such as implementation, policy, impact, or place statistics.
- Knowledge repository: Stores the annotated information for use by downstream services.
Outcome
The Knowledge Discovery Engine was the backbone of Urbanixm’s value proposition, both as a standalone SaaS and as a case-study research service.
Technology
Python, LangChain, retrieval-augmented generation (RAG), large language models (LLMs), information retrieval