Sr. Data Engineering Manager- Platform & Streaming & Cloud

Company: U.S. Bank
Location: San Francisco
Posted on: April 10, 2021

Job Description:

Only Local Bay Area candidates employed as FTE at Large Tech, FAANG, Start up's will be considered.Open for H1b Transfer/ Salary No bar for the right talent.Note: Consulting/Third party candidates will not be considered, please apply to other roles at our Careers page!!At U.S. Bank, we're passionate about helping customers and the communities where we live and work. The fifth-largest bank in the United States, we're one of the country's most respected, innovative and successful financial institutions. U.S. Bank is an equal opportunity employer committed to creating a diverse workforce. We consider all qualified applicants without regard to race, religion, color, sex, national origin, age, sexual orientation, gender identity, disability or veteran status, among other factors.Job DescriptionSenior Big Data Engineer will design, develop and implement data pipelines ,platforms and customer facing products using Big Data technologies in an agile environment. Senior Big Data Engineer will partner with cross functional teams and business stakeholders to drive the SDLC , shape the quality of product and deliver in a timely manner. Senior Big Data Engineer is also responsible for monitoring and troubleshooting of the data products / related systems and help improve the operational/functional efficiency.Role And Responsibilities* Design and build highly available, scalable, resilient and fault tolerant distributed real-time data streaming system to collect, enrich and distribute to multiple Destinations.* Work simultaneously on multiple projects and prioritize them with cross collaboration with stakeholders* Drive larger initiatives in the data space from concept inception ,analysis to on-boarding .* Help mentor and coach junior engineers on the best practice ,work culture etc., by being an example* Collaborate cross-functionally with product owner, data scientists, business users, project managers and other engineers to achieve elegant solutions.* Build real time streaming pipelines that deliver data with measurable quality under the SLA* Drive efforts to improve the data quality across data pipelines and implement system controls for managing data quality* Automate test coverage (90+%) for data pipelines, best practices and frameworks for unit, functional and integration tests.* Champion best practices for high availability, scalability and reliability of data streaming components.* Build AI/ML model based anomaly detection algorithms to detect data quality issues and abnormal behavior across different real time data sets/events.Basic QualificationsBachelor's degree or equivalent work experience.5+ years of relevant experience.Preferred Skills/Experience* 10+ years of overall experience in software development with 5-6 years of relevant experience in designing, developing, deploying and operating data streaming pipelines at scale.* 3-4 years' experience with Apache Kafka and Apache Spark Streaming* Experience in Spark Structured Streaming preferred.* Experience in tuning Spark Data pipelines to achieve high throughput with low latency.* Programming proficiency in Scala or Java.* Open Source Committer (Apache Spark or related Big Data open source technologies) is preferred.* Experience with Containers, Kubernetes and scaling elastically* Strong background in algorithms and data structures and continuously develop and acquire new technical skills* Experience providing technical leadership and mentoring other engineers for best practices on data engineering* Experience in automating Spark pipeline deployment/testing (DevOps, CI/CD)* Knowledge of various databases / database technologies - Cassandra (NoSQL), Postgres, or other columnar databases.* Passion for data engineering and for enabling others by making the product easier to use.* Collaborate cross-functionally with product owner, data scientists, business users, project managers and other engineers to achieve elegant solutions.* Excellent communication in sharing context to effectively collaborate with analytical partners, domain experts and other consumers of your work, preferably in supporting an engineering or product function* Required Skills Summary:--Apache Spark, Apache Kafka, Scala/Java, NoSQL Databases, Elasticsearch & Kibana, Kubernetes, Docker ContainersBenefitsTake care of yourself and your family with U.S. Bank employee benefits. We know that healthy employees are happy employees, and we believe that work/life balance should be easy to achieve. That's why we share the cost of benefits and offer a variety of programs, resources and support you need to bring your full self to work and stay present and committed to the people who matter most - your family.Learn all about U.S. Bank employee benefits, including tuition reimbursement, retirement plans and more, by visiting usbank.com/careers .EEO is the LawApplicants can learn more about the company's status as an equal opportunity employer by viewing the federal EEO is the Law poster.E-VerifyU.S. Bank participates in the U.S. Department of Homeland Security E-Verify program in all facilities located in the United States and certain U.S. territories. The E-Verify program is an Internet-based employment eligibility verification system operated by the U.S. Citizenship and Immigration Services. Learn more about the E-Verify program .

Keywords: U.S. Bank, San Francisco , Sr. Data Engineering Manager- Platform & Streaming & Cloud, Executive , San Francisco, California