Staff Software Engineer, SWAP

We're looking for an experienced Software Engineer to join our Spark Workload Acceleration Platform (SWAP) team. The SWAP team builds core infrastructure for accelerating large-scale data processing and machine learning workloads on Databricks. As a Staff Software Engineer on the team, you'll be responsible for designing and building highly scalable and reliable systems that are critical to the success of our customers. Your Responsibilities: - Design and build fault-tolerant, highly available, and scalable distributed systems to accelerate big data workloads - Implement robust monitoring and observability solutions to ensure reliability and performance - Contribute to technical roadmaps and drive the implementation of new features and capabilities - Mentor junior engineers and share your expertise to help grow the team - Collaborate cross-functionally with various engineering teams to deliver end-to-end solutions Qualifications: - 8+ years of software engineering experience building large-scale distributed systems - Expertise in Java, Scala, or Python and comfortable working with big data technologies like Spark, Hadoop, Kafka, etc. - Strong understanding of system design, data structures, and algorithms - Experience with building highly available, fault-tolerant, and scalable systems - Proficient in designing and building reliable, observable, and maintainable infrastructure - Excellent problem-solving and analytical skills - Strong communication and collaboration skills