Spark Software Engineer

Company: CYNET SYSTEMS
Location: San Francisco
Posted on: January 26, 2023

Job Description:

Job Description:

Pay Range: $55 - $60

Responsibilities:

• 
  
• Prepare training material and hold technical training sessions for internal Spark users.
  
• Developing Spark infrastructure, Spark user libraries and tooling to make direct impact to users.
  
• Interact with multiple groups on a daily basis to design data solutions and promote the best Spark practices among users.
  
• Solve production issues related to user Spark applications.
  
  Basic Qualifications:
  
  
  • 
    
  • BS/MS in Computer Science or related technical discipline.
    
  • 1+ years of relevant work experience in Spark and Hadoop.
    
  • Excellent communication and presentation skills.
    
  • Good understanding of Spark, Hadoop, HDFS, YARN, Hive technologies.
    
  • Good understanding of distributed storage and computing systems.
    
  • Good understanding of programming languages like Java and Scala.
    
  • Good understanding of scripting languages like Shell and Python.
    
    Preferred Qualifications:
    
    
    • 
      
    • Good understanding in Spark Core and Spark SQL engine.
      
    • Experience in building large scale applications with Spark SQL.
      
    • Experience in writing, analyzing, and debugging SQL queries using Hive and Spark SQL.
      
    • Experience with tuning large scale Spark jobs.
      
    • Experience in working with a data format on Hadoop/Spark like Avro, Orc, and Parquet.
      
    • Experience with Interactive Spark Notebooks (Jupyter Notebooks).
      
      

Keywords: CYNET SYSTEMS, San Francisco , Spark Software Engineer, IT / Software / Systems , San Francisco, California