DV
Back to all projects

Denver Urban Pulse

Data Pipeline & BI Analytics Platform

Automated analytics platform using data from 4 public APIs, a PostgreSQL raw → staging → marts architecture, Python ETL, interactive maps, heatmaps, KPI trends, and centralized filtering. Data is refreshed on a scheduled batch pipeline, with source freshness shown in the dashboard.

SQL PostgreSQL Python ETL API Integration Data Modeling Next.js TypeScript Leaflet Recharts
Denver Urban Pulse

Overview

Automated analytics platform using data from 4 public APIs, a PostgreSQL raw → staging → marts architecture, Python ETL, interactive maps, heatmaps, KPI trends, and centralized filtering. Data is refreshed on a scheduled batch pipeline, with source freshness shown in the dashboard. It covers crime, traffic crashes, 311 service requests, and air quality, with neighborhood-level analysis.

Architecture

┌─────────────────────────────────────────────────────┐
│                    Data Sources                      │
│  ArcGIS (Crime, Crashes, 311)  ·  AirNow API (AQI)  │
└──────────────────────┬──────────────────────────────┘
                       │  daily cron (06:00 UTC)
                       ▼
┌─────────────────────────────────────────────────────┐
│              Python Pipeline (Railway)               │
│  Migrations → Ingestion → Staging → Marts            │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│              PostgreSQL (Railway)                     │
│  raw (5 tables) → staging (5) → marts (9)            │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│          Next.js API Routes (Vercel)                 │
│  /city-pulse/kpis  · /categories · /heatmap          │
│  /category-trends  · /neighborhoods                  │
│  /environment/aqi  · /environment/comparison          │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│            React Dashboard (Vercel)                   │
│  KPIs · Map · Charts · Heatmap · Ranking · Filters   │
└─────────────────────────────────────────────────────┘

Data Sources

SourceProviderRefresh
Crime incidentsDenver Open Data (ArcGIS)Daily, last 90 days
Traffic crashesDenver Open Data (ArcGIS)Daily, last 90 days
311 service requestsDenver Open Data (ArcGIS)Daily, last 90 days
Air quality (AQI)AirNow APIDaily, last 90 days
Neighborhood boundariesDenver Open Data (GeoJSON)On change

Key Features

Tech Stack

LayerTechnology
DatabasePostgreSQL (Railway)
Data PipelinePython 3.12 daily cron (Railway)
FrontendNext.js 16 (App Router), React 19, TypeScript 5.9
StylingTailwind CSS 4, shadcn/ui, IBM Plex Sans
ChartsRecharts 3
MapLeaflet + React Leaflet
APINext.js Route Handlers (7 endpoints)
DeployVercel (frontend), Railway (DB + pipeline)
TestingJest 30, React Testing Library