Senior Software Engineer, Data Platforms

Job Description

At Lyft, our purpose is to serve and connect. We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. We are seeking a talented Software Engineer to join our dynamic Streaming Compute Team. In this role, you'll be instrumental in designing, developing, and maintaining our low latency and high throughput systems, which are critical to a lot of teams like driver, mapping, fraud etc. and almost all ML models at Lyft. Our systems and platform are used for a variety of use cases such as, persisting high volume service events, detecting issues with routes, computing prime-time, creating, storing and serving ML features for training and inference purposes for teams like payments, incentives etc. The majority of our streaming solutions run on Apache Flink and Apache Beam with support from reading and writing to various sources and sinks respectively. As a Software Engineer, you will lead the team’s projects and initiatives seeing them to completion. You will be responsible for key systems and components within the team. You will identify and lead the efforts in improving architecture and operations for the team. We are looking for candidates who have prior experience with near-real-time streaming systems, are self-starters, and have a proven track record of delivering high-quality software solutions that solve critical business needs. The candidate should be able to dive deep into problems with lots of ambiguity, leading architectural design and building robust, well-tested technical solutions. They should be willing to take complete ownership of a project or a feature, able to drive it from design and implementation through to operational excellence, while also mentoring others, improving team engineering practices, and collaborating effectively with partners. Responsibilities: Design, develop, deploy, monitor, operate and maintain scalable streaming solutions that constitute our streaming platform Work with technologies such as Apache Kafka, Flink, Beam etc. to enhance our data platform's streaming offerings Collaborate with cross-functional teams to integrate streaming solutions into the broader data architecture Analyze our internal systems and processes and locate areas for improvement/automation Collaborate with product org stakeholders to address and prioritize custom edge cases Help lead large projects from inception to positive execution Operate a real-time streaming platform with high availability and low downtime to meet the demands of latency-sensitive workloads. Experience: 5+ years of experience with building, deploying and maintaining low latency, high throughput services 5+ years of experience building and developing large-scale infrastructure, distributed systems or networks, and/or experience with compute technologies 3+ year

Required Skills