Data Engineer (Healthcare Map Team)

Company: Komodo Health
Location: San Francisco
Posted on: June 13, 2021

Job Description:

We Breathe Life Into Data!

At Komodo Health, our mission is to reduce the global burden of disease. Smarter use of data is essential to this mission. We combine the world's most comprehensive view of patient encounters with innovative algorithms and decades of clinical expertise to power our Healthcare Map, the industry's most precise view of the U.S. healthcare system. With the Healthcare Map as our foundation, we offer a suite of powerful software applications that deliver exceptional value to the industry.

Our people are the center of our success. We are a smart, supportive team with diverse perspectives and a shared passion for fixing what's broken in healthcare. It's fun, challenging and important. Join us!

The Opportunity at Komodo Health

Komodo aims to build the best healthcare data architecture in the industry. As we are maturing, we are investing in our core data assets, and continually expanding the number and diversity of the data sources we are ingesting.

Our Data Ingestion team is a high-performing team working with big data at scale - they both ingest data and evolve the architecture to increase efficiency. The team is currently starting work on version 2.0 of their architecture, focused around improving efficiency, reducing operations and improving data quality. Our data comes in daily, meaning we do batch processing and not streaming.

This role will be a key contributor to the design and implementation of data engineering pipelines to ingest billions of medical and prescription claims, as well as other data sources. Our data ingestion use cases are complex (hundreds of fields, many come as multiple data files that need to be linked), and implementing robust data quality processes and frameworks is crucial.

Looking back on your first 12 months at Komodo Health, you will have

• Owned ingestion of several complex data sources and / or enhanced existing data sources.
• Participated in design and implemented distributed data systems using Spark or similar technologies.
• Collaborated with product managers and data scientists to implement validation and/or data quality enhancements.
• Optimized the pipelines for performance and cost.
• Helped ensure smooth operations of the ingestion pipelines.
• Helped ensure patient privacy by implementing business rules.

Some of the projects the team is currently working on:

• Building new architecture focused on improving consistency between different pipelines, improving ability to backfill, and increasing data quality.
• Ingesting new complex data sources. Many of our data sources consist of a large historical load, followed by regular incremental deliveries.
• Enabling data ingestion as a service - to enable other teams to self-serve their ingestion needs when they are less sophisticated

What you bring to Komodo:

• Expertise with industry standard distributed systems (ie. Spark), data pipeline tools (ie. Airflow).
• Capable of quickly building expertise on an as-need basis on new tech stack.
• Demonstrable experience with Python.
• Understand and design for non-functional concerns such as performance, cost optimization, maintainability and developer experience.
• Strong communication with engineers, product managers.
• Ability to work as part of an agile collaborative team in a fast-paced environment.
• Experience with systems support, debugging and operations.

Curiosity piqued? Learn more about us!

• Komodo Health Secures $220M in Series E funding led by Tiger Global
• Komodo Health Acquires Mavens, Series D funding led by ICONIQ Growth
• Wealthfront: The 2021 Career Launching Companies List
• In Conversation with Dr. Arif Nathoo, Komodo Health's CEO
• Komodo Health's 'Meet a Dragon' series
• Komodo's Values that Drive our Culture

Keywords: Komodo Health, San Francisco , Data Engineer (Healthcare Map Team), Other , San Francisco, California