Sr Software Engineer -Cortex
Company: Twitter
Location: San Francisco
Posted on: June 13, 2021
|
|
|
Job Description:
The Position Who we are Cortex empowers internal teams to efficiently leverage ML by
providing a platform and by unifying, educating, and advancing the
state of the art in ML technologies within Twitter. We win when our customers win by helping our users stay
informed, share and discuss what matters; by serving the public
conversation. We're building an AI-first company and every major
initiative is increasingly dependent on the successful application
of machine learning. Cortex is at the nexus of this evolution. Our
team of ML software engineers is constructing one of the strongest
machine learning platforms in the world, based on the latest ML
industry practices, deep learning, engineering excellence, and
powered by Twitter data at scale. The ML Feature Management (MLFM) team is part of the Cortex
Platform group. Together with the other four Cortex Platform teams,
we develop tools and infrastructure that standardizes machine
learning development at Twitter on a common platform. This allows
ML engineers in our customer teams to do their work better and
faster and with a more modern stack that follows broad industry
trends. More specifically, our team's vision is to maximize the velocity
of Feature Engineering. To date, we've built and heavily invested
in Twitter's Feature Store, to share features across different ML
product teams in production and thereby to broaden the impact of
feature engineering investments across the company. This has been a
wildly successful mission over the last two years. The Feature
Store is running in production in many of Twitter's ML stacks,
serving millions of requests per second at super low latencies
which has impacted top-line metrics like daily active users. These
wins have prompted all remaining ML teams to migrate to the Feature
Store. We're assisting with this migration which we expect to
complete in 2021. Our other investments for 2021 include: Core API improvements include new functionality to deliver a
more complete solution to our customers. Simplifying our architecture with an eye toward expandability
and maintainability. Systematic performance optimization and increasing production
resilience. Automating workflows for feature registration and for migration
to the Feature Store as much as possible. Support new and different data formats for feature data. We're participating in Cortex's virtual team to explore ML
tooling available in the Google Cloud Platform (GCP). For our team,
this means integrating data processing engines such as BigQuery and
Dataflow and their application to feature engineering use
cases. Prototyping tooling to accelerate feature experimentation and
exploration. Building ML-as-a-service, a system that can handle dataset
creation, training and online feature hydration purely from
configurations and model specification, with the purpose of taking
successful notebook experiments to an A/B test in production
quickly. Systematically tracking and analyzing metadata of all stages of
tooling for answering questions around feature-model lineage, data
privacy protection and easier model performance debugging. What you will do We're looking for a senior engineer to join us in our core
Feature Store workstream who can help us simplify the code base
while adding new core functionality to tackle complex use cases,
but without compromising performance. If thinking in architectural
frameworks, like primitives, design patterns, functional paradigms,
or the commonality-variability principle is your middle name, then
we want to talk to you. If chasing down latency losses, drilling
into performance traces and dissecting the JVM memory model is up
your alley, give us a call. Qualifications Who you are Do you identify with the majority of the following traits? Yes?
We believe they will make you successful in this role. You're fluent in concepts like design patterns, composition vs
inheritance, commonality variability principle, primitives,
etc. You have a solid grasp on type systems and functional
programming, ideally in Scala or Java and a dynamic language like
Python. You have educated opinions about what good library and API
design in distributed systems looks like, both from formal study
and from lived experience with past design decisions you made. You're able to articulate such opinions and concepts
persuasively both in spoken and written form, with the intent to
teach and enroll others, while remaining open to a constructive
dialogue. Your thought process intrigues others with its
clarity. You take a principled, disciplined approach to software
engineering, characterized by high volume and quality, solid
testing practices, data-driven decision making, and a low rate of
incidents in production. You take pride in the developer experience, the ease of use,
documentation and the durability of the products you build,
together with the team. You have a passion for machine learning and high-leverage
infrastructure platforms. You bring partners together across organizational and functional
boundaries. By nature of the problem domain, we expect you to have
experience in: 5+ years building and delivering working software through an
iterative, agile process. 2+ years in Sr engineering or architect capacity with a track
record of resilient design decisions shipped to production. 4+ years of work experience in software engineering in the areas
of distributed data processing, in high traffic or large data
production environments. 2+ years with large Scala or Java code bases with many
contributors Company Description Twitter is what's happening and what people are talking about
right now. For us, life's not about a job, it's about purpose. We
believe real change starts with conversation. Here, your voice
matters. Come as you are and together we'll do what's right (not
what's easy) to serve the public conversation.
Keywords: Twitter, San Francisco , Sr Software Engineer -Cortex, Other , San Francisco, California
|
Click
here to apply!
|
