SanFranRecruiter Since 2001
the smart solution for San Francisco jobs

Software Engineer, Data Platform

Company: SmartNews, Inc.
Location: San Francisco
Posted on: April 11, 2021

Job Description:

SmartNews is a leading mobile app of news aggregation services. We analyze millions of articles to deliver the most engaging information with high quality, in near-real time fashion, to millions of users around the world. The data platform software engineer in SmartNews plays a key role in accelerating the products/business developments. Great efforts are paid on building a highly efficient and flexible data service for analytical and operational purposes.
To serve the internal users from analytics and product-dev teams, the goal and mission of data engineers is to create high-level, easy-to-use data services for simplifying the accessing, integration and consolidation of various data sets, and also building the platforms for executing tasks processing massive data in terms of TB per day. Technology drives the growth of SmartNews, and thus we eagerly adopting cutting-edge technologies from the industry and academia especially, the open-source community.
ResponsibilitiesThis is a hybrid of system engineering and machine learning role:

  • Design, develop, setup and maintain new services, libraries, tools, frameworks for data processing or management, and investigate new algorithms to increase efficiency for Data Processing, such as ETL, Data Pipelines, OLAP DBMS, real-time messages and streams processing, data-sync between systems, etc.
  • Develop in-house
  • Develop toolings for system performance evaluation, monitoring and tuning of the data processing procedures or platforms, get insights of efficiency and stabilizability and make continuous improvement, such as optimizing distributed query engines, computing resource management and isolation, multi-tier storage systems, etc.
  • Own and maintain the key data processing portfolios such as building and taking care of the environment, trouble-shooting and being responsible to the on-call of incidents. Work closely with data architecting/modeling roles to understand ways to implement the data service, and interact with Site Reliability Engineering (SRE) team to deploy the environments and drive production excellence.
  • Device system, tooling and approaches for data privacy and security. Establish access control, create processes to handle sensitive data.
  • Diagnose and resolve complex technical challenges for data accessing or processing. Using elegant and systematic rather than ad-hoc methods to help other teams tuning the performance and improving stability.
    Qualifications
    Minimum Requirements
    • BS/MS degree in computer science or equivalent science/engineering degree with 5 or more years of experience
    • Strong Programming skills and experiences with deep understanding of data structures and algorithms are required for building efficient and stable solutions
    • Rich experiences with one or more programming languages such as Java, Scala, C++ or Python; familiar with agile development and manage testing skills
    • Need certain knowledge on shell scripts and operating systems, especially on Linux
    • Good understanding of modern bigdata technologies and ecosystems
    • Familiar with Hadoop, Spark, Hive, Presto, Redis and Flink, be able to develop data processing programs with them in batch or streaming manner
    • Familiar with modern data stores either RDBMS or NoSQL stores (such as MySQL, Cassandra or Druid, etc); have experiences on developing application or function-extensions on such data stores
    • Be able to implement and tune complicated heavy-lifting data flows (ETLs or pipelines), familiar with certain toolings
    • Capability of system design with good modularity and extensibility
    • Familiar with system/module design methods and toolings such as UML
    • Be able to draft the user-understandable blueprint and precise, detailed designs
    • Experience of building highly scalable distributed systems
    • Able to design and implement distributed services with scalability and performance in mind
    • Able to debug and troubleshooting performance and reliability problems

    • Preferred Requirements
      • Experience with cloud based architecture (e.g. Amazon Web Services)
      • Strong interest in high performance, high availability, hive volume data processing
      • Good experience in data management, integration, security, and auditing
        Benefits
        • Equity included
        • 100% medical, dental and vision insurance coverage (60% coverage for dependents)
        • Monthly housing, commute, mobile phone and gym allowances
        • 401k matching program
        • Free lunch, snacks, drinks, etc.
        • Pet friendly office

Keywords: SmartNews, Inc., San Francisco , Software Engineer, Data Platform, IT / Software / Systems , San Francisco, California

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category
within


Other IT / Software / Systems Jobs


Programmer Analyst 4-IT
Description: Work with Oracle's world class technology to develop, implement, and support Oracle's global infrastructure. As a member of the IT organization, assist with the analyze of existing complex programs and (more...)
Company: Oracle
Location: Newark
Posted on: 04/12/2021

Mobile Engineering Manager
Description: Join Hired and find your dream job as a Mobile Engineering Manager at one of 10,000 companies looking for candidates just like you.Companies on Hired apply to you, not the other way around. You'll (more...)
Company: HIRED
Location: San Bruno
Posted on: 04/12/2021

Tooling and Infrastructure Software Engineer - Python
Description: Tooling and Infrastructure Software Engineer - Python in Santa Cruz, California - Careers at Joby Aviation Please Enable Cookies to Continue Please enable cookies in your browser to experience all the (more...)
Company: Joby Aviation
Location: Santa Cruz
Posted on: 04/12/2021


Software Engineer (Security Clearance Required) - Remote
Description: Join Hired and find your dream job as a Security Software Engineer at one of 10,000 companies looking for candidates just like you.Companies on Hired apply to you, not the other way around. You'll receive (more...)
Company: Hired
Location: West Sacramento
Posted on: 04/12/2021

Software Engineer (Security Clearance Required)
Description: Join Hired and find your dream job as a Security Software Engineer at one of 10,000 companies looking for candidates just like you.Companies on Hired apply to you, not the other way around. You'll (more...)
Company: HIRED
Location: Rohnert Park
Posted on: 04/12/2021

Junior Software Developer
Description: Revature is the fastest growing employer of emerging technology talent. We will hire over 300 new Junior Software Developers in the next 4 weeks.One day someone is going to ask you where you got your (more...)
Company: Revature
Location: Santa Cruz
Posted on: 04/12/2021

Simulation Software Engineer ADAS/AD
Description: Leading the future of luxury mobility br Lucid's mission is to inspire the adoption of sustainable energy by creating the most captivating luxury electric vehicles, centered around the human experience. (more...)
Company: Lucid Motors
Location: Newark
Posted on: 04/12/2021

Software Development Engineer in Test
Description: Software Development Engineer in Test in Santa Cruz, California - Careers at Joby Aviation Skip to Main Content Please Enable Cookies to Continue Please enable cookies in your browser to experience all (more...)
Company: Joby Aviation
Location: Santa Cruz
Posted on: 04/12/2021

Software Architect
Description: Dynamic Signal is the leading Employee Communication and Engagement Platform, committed to creating a connected, inclusive, and engaged workforce where people feel valued and empowered to be their best. (more...)
Company: IGN Entertainment, Inc.
Location: San Bruno
Posted on: 04/12/2021

Technical Writer
Description: Senior Technical Writer Fixed Term Contract Five9 is a leading provider of cloud software for the enterprise contact center market, bringing the power of the cloud to thousands of customers and facilitating (more...)
Company: Five9
Location: San Ramon
Posted on: 04/12/2021

Log In or Create An Account

Get the latest California jobs by following @recnetCA on Twitter!

San Francisco RSS job feeds