Back to Projects
AI/Data Platform

Real Estate AI Data Agent

Modern Data Ecosystem for National Newspaper

Matter Product Studio Inc.2023-2024
AI Data Agent Architecture

Overview

Led distributed team to transform a national newspaper's underutilized 210GB+ real estate data into a highly efficient AI Data Agent, delivering real-time business signals and actionable insights through custom APIs and modern data infrastructure.

Challenge

A newspaper organization was sitting on a wealth of data but lacked technical expertise and infrastructure to organize, manipulate, and extract meaningful insights. They had no strategy for managing data, no API structure for current or future projects, and needed to transform raw data into usable business signals accessible to development teams.

My Role

Senior Technical Product Manager - Led distributed team, developed comprehensive data strategy, facilitated discovery workshops with 23 stakeholders across 7 departments, managed API-first architecture, data governance, CI/CD, and defined roadmaps for multiple teams.

Approach

Leveraged Snowflake and dbt to clean and structure existing data with continuous updates. Built two key APIs: (1) Real-time Kafka event-streaming API for business transactions and property records tied to individuals/businesses, (2) Poll API for detailed query-based access to specific records. Organized data into categories connecting people, businesses, and properties to generate actionable business signals.

Technology Stack

SnowflakedbtKafkaCustom APIsGitHub ActionsCensusGraphQLHasuraAWS

Key Outcomes

Transformed underutilized data resource into highly efficient, data-driven organization
210GB+ data centralized in Snowflake for scalable cloud storage
Real-time event streaming API using Kafka for continuous business signals
Poll API for on-demand detailed record access
1M+ daily data changes processed
Foundation established to support multiple teams and future projects
Positioned client for future success with actionable insights

Project Gallery