Spark Eighteen
Website:
sparkeighteen.com
Job details:
We are seeking a Data Engineer with 5+ years of experience to build and scale data platforms powering AI-driven analytics on high-volume sensor, satellite, and third-party data. The ideal candidate will have expertise in designing and managing scalable batch and real-time data pipelines using Apache Airflow, Apache Spark, Kafka, Python, and SQL. Strong experience with PostgreSQL, TimescaleDB, PostGIS, cloud-based data lake architectures, and geospatial/time-series data is essential. The role involves owning data pipelines end-to-end from real-time ingestion to analytics- while ensuring data quality, performance, observability, and reliability in a cloud-native environment. Strong stakeholder management, problem-solving, and collaboration skills are key to delivering impactful data products and analytical solutions.
Requirements
- 5+ years of experience in Data Engineering and production-scale data platforms
- Design and manage Apache Airflow pipelines for high-volume sensor, satellite, and third-party data ingestion
- Build and optimize Apache Spark workloads for batch processing, geospatial analytics, and large-scale aggregations
- Develop and maintain PostgreSQL, TimescaleDB, PostGIS, and DuckDB-based data solutions
- Implement Kafka-based streaming pipelines for real-time data ingestion
- Work with Object Storage (AWS S3 or equivalent) for scalable data lake architecture
- Strong proficiency in Python, SQL, Airflow, Spark, and PostgreSQL
- Experience with geospatial and time-series data
- Comfortable working in Ubuntu-based environments
- Collaborate with customer teams to translate business requirements into data products and analytical solutions
- Ensure data quality, lineage, observability, performance, and SLA compliance
- Troubleshoot production issues and optimize pipeline, database, and query performance
- Create and maintain technical documentation, data models, and run-books
- Strong communication, stakeholder management, and problem-solving skills
Benefits
- Comprehensive insurance coverage that gives you peace of mind, so you can focus on doing your best work
- Flexible work arrangements designed to support sustained productivity, personal well-being, and work-life balance
- Continuous learning and accelerated skill development through hands-on projects and mentorship from experienced industry leaders
- Global client exposure across 20+ countries, offering real-world experience with diverse markets and business environments.
- Opportunity to work on high-impact, large-scale projects that have collectively generated over $1B in measurable business value
- Competitive, market-aligned compensation packages that recognize performance, expertise, and long-term contribution
- Monthly demo days that celebrate innovation, showcase your work, and give you a real voice in what we build
- Annual recognition programs and performance-driven awards in a truly meritocratic environment
- Referral bonuses that reward you for helping grow a strong, like-minded team
- A strong problem-solving culture with opportunities to tackle meaningful, real-world challenges
- A positive, people-first workplace that supports happiness, balance, and long-term growth
Click on Apply to know more.
