Truecar is an online car purchasing platform that allows users to communicate with TrueCar Certified Dealers for a great car buying experience. I was on the Data Engineering Team, which is responsible for the development of data pipelines that ingest, analyze, and validate automotive data from internal and 3rd party sources.

DataOne Pipeline Development

For this assignment, I was responsible for the development, integration and testing of a brand new data pipeline to process and validate records from our new data vendor, DataOne. This pipeline processes thousands of new records each day for Ford and Acura vehicles, allowing for much more accurate price estimations and car configurations

Pipeline Optimizations

I was tasked with diagnosing and remedying slow aspects of the pipeline responsible for the ingestion of all new car data records listed on the marketplace. Through various optimizations and improvements, I was able to cut overall execution time of this pipeline down 20%, allowing for this pipeline to run more frequently and efficiently