Lead Data Engineer, Disney Streaming

Company: The Walt Disney Company
Location: San Francisco
Posted on: January 16, 2022

Job Description:

As a Lead Data Engineer you will be designing, building and supporting data pipelines consuming data from multiple different source systems and transforming it into valuable and insightful information. You will have the opportunity to contribute to end-to-end platform design for our cloud architecture and work multi-functionally with operations, data science and the business segments to build batch and real-time data solutions.

The role will be part of a team supporting our Corporate and Segment business lines, including Networks (ABC and ESPN) Studios (Disney, Pixar, Marvel, LucasFilm), Parks & Resorts and Disney Consumer Products & Interactive.

Responsibilities :

• Design, develop and implement large-scale, high-volume, high-performance data lake and data warehouse.
• Build ETLs/ELTs to take data from various data sources and create a unified/enterprise data model for analytics and reporting.
• Build, deploy and support batch & real-time, fault tolerant, self-healing data pipelines
• Work in close collaboration with product management, peer system and software engineering teams to clarify requirements and translate them into robust, scalable, operable solutions that work well within the overall data architecture
  
  Basic Qualifications :
  
  • Experience with workflow orchestration tools like Airflow (2+ years)
  • Deep experience of relational, analytical, columnar databases (7+ years)
  • Proficiency in Hadoop ecosystem, experience writing MapReduce, Hive or Spark jobs (3+ years)
  • Programming in Python or Java. (3+ years)
  • Proven strength in SQL, data modeling, ETL development, and data warehousing (7+ years)
  • Experience with developing and deploying in a cloud native environment preferably AWS cloud. (3+ years)
  • Design solutions for structured and unstructured data
  • Experience with Git, Continuous Integration and related tools
  • Good unit testing and integration testing skills
  • Experience with workflow orchestration tools
  • Experience delivering in a team environment using SAFe Agile framework
    
    Preferred Qualifications:
    
    • Experience with AWS Kinesis, or Google Pub/Sub
    • Experience with BI tools like Tableau etc.

Keywords: The Walt Disney Company, San Francisco , Lead Data Engineer, Disney Streaming, Engineering , San Francisco, California