Data Engineer

Company: Grid Dynamics International
Location: San Francisco
Posted on: November 22, 2021

Job Description:

Big Data Development San Francisco, California Description Position at Grid Dynamics
Responsibilities:--

• Develop and manage ETLs to source data from various sales and operational systems and create unified data model for analytics and reporting
• Design, implement, and support data warehouse / data lake infrastructure using Azure big data stack, Python, Spark etc
• Maintain and support existing platforms and evolve to newer technology stacks and architectures
• Creating ETL/Ingestion jobs using Apache Spark and monitor/maintain the jobs on Cloud (Azure)
• Identify and surface performance and optimization oppurtunities and implement caching to improve the performance of derived views and using different joint methodologies like hash, nested, and merge.Requirements:
  
  • Bachelor's degree in computer science or equivalent experience and 5+ years of softwre development experience
  • Strong experience working on spark platform, Python and python libraries
  • Working experience on cloud-based environments such as Databricks on cloud platform environment, ADF, PySpark etc
  • Proficiency writing notebooks using Python/Spark
  • Automation using Linux shell & Python scripting
  • Hands-on experience in the DevOps model, production support and Agile methodologies--
  • Possess strong analytical and problem-solving skills and proven ability to think objectively and interpret meaningful themes from quantitative and qualitative data
  • Experience with Azure is a plusWhat we offer:
    • Opportunity to work on bleeding-edge projects
    • Work with a highly motivated and dedicated team
    • Competitive salary
    • Flexible schedule
    • Benefits package - medical insurance, sports--
    • Corporate social events
    • Professional development opportunities
    • Well-equipped office-- About Us:-- Grid Dynamics is a leading provider of technology consulting, agile co-creation, scalable engineering and data science services for Fortune 500 corporations undergoing digital transformation.-- We work in close collaboration with our clients on digital transformation initiatives that span strategy consulting, early prototypes and enterprise-scale delivery of new digital platforms. We help organizations become more agile and create innovative digital products and experiences using deep expertise in emerging technology, top global engineering talent, lean software development practices, and high-performance product culture.-- Headquartered in Silicon Valley with over 1,300 technologists located in engineering delivery centers throughout the US, Central and Eastern Europe, Grid Dynamics has architected and delivered some of the most extensive digital transformation programs in the retail, technology and financial sectors to help its clients win market share, shorten time to market and reduce costs of digital operations on a massive scale. To learn more about Grid Dynamics, visit-- www.griddynamics.com , or follow us on Twitter @GridDynamics. You have been redirected to a Grid Dynamics job page

Keywords: Grid Dynamics International, San Francisco , Data Engineer, Engineering , San Francisco, California