Website:
dreampathservices.com
Job details:
Job Role: Senior Data Engineer
Experience: 7+ Years
Location: Hyderabad Onsite
Employment Type: Full-Time
Evening shift starts at 6:30 Pm
Job Summary
We are seeking an experienced Senior Data Engineer with strong expertise in AWS, DBT, Databricks, PySpark, Airflow, and SQL to build and scale modern data platforms. The ideal candidate will have a proven track record of designing, developing, and optimizing large-scale data pipelines and data models that support analytics, machine learning, experimentation, and business reporting.
This role requires exceptional communication skills, a strong ownership mindset, and the ability to collaborate closely with Data Scientists, Analytics teams, and business stakeholders to deliver reliable, high-quality data solutions.
Key Responsibilities
- Design, develop, and maintain scalable DBT models that deliver trusted datasets, business metrics, and analytical features.
- Build and manage data pipelines using Databricks and PySpark to transform raw operational data into analytics-ready datasets.
- Develop and optimize data lakes and lakehouse architectures using Delta Lake and/or Apache Iceberg.
- Orchestrate end-to-end workflows using Apache Airflow and Prefect, ensuring high reliability and SLA adherence.
- Partner with Data Science teams to design, develop, and productionize feature engineering pipelines.
- Implement data quality frameworks, automated testing, monitoring, and documentation standards.
- Optimize SQL queries, Spark jobs, and DBT transformations for performance, scalability, and cost efficiency.
- Design and manage cloud-native data solutions within the AWS ecosystem.
- Participate in code reviews and contribute to engineering best practices.
- Build deep business understanding to ensure data products accurately represent operational processes.
- Leverage AI-powered development tools such as Cursor, GitHub Copilot, Claude Code, or similar to improve engineering productivity.
Required Skills
Must-Have Technologies
- DBT (Data Build Tool)
- AWS (Strong Hands-on Experience)
- Databricks
- PySpark
- Apache Airflow
- Advanced SQL
Additional Technical Skills
- Delta Lake / Apache Iceberg
- Python
- Data Modeling & Data Warehousing
- Distributed Data Processing Systems
- Infrastructure as Code (Terraform, AWS CDK, or Pulumi)
- CI/CD for Data Engineering
- Data Quality & Testing Frameworks
- ETL/ELT Architecture
Required Experience
- 7+ years of experience in Data Engineering.
- Strong experience designing and deploying large-scale data pipelines in production environments.
- Hands-on experience with distributed computing systems and concepts such as consistency, fault tolerance, throughput, and latency.
- Proven expertise in building scalable ELT/ETL frameworks using DBT, Databricks, Spark, and Airflow.
- Experience working within AWS-based data ecosystems.
- Exposure to modern lakehouse architectures and big data technologies.
- Experience supporting analytics, business intelligence, and machine learning workloads.
Preferred Qualifications
- Experience with Kinesis, EMR, Sigma, or Pulumi.
- Exposure to supply chain, logistics, transportation, or operations-focused data domains.
- Familiarity with feature stores, MLOps, and production ML pipelines.
- Experience working in fast-paced, high-growth environments.
Soft Skills
- Outstanding verbal and written communication skills.
- Strong stakeholder management and collaboration abilities.
- Self-driven, proactive, and capable of working independently.
- Strong problem-solving and analytical mindset.
- Ability to quickly learn and adapt to new technologies.
- Comfortable mentoring junior engineers and driving engineering excellence.
Keywords
AWS, DBT, Databricks, PySpark, Airflow, SQL, Delta Lake, Apache Iceberg, Data Engineering, Data Modeling, ETL, ELT, Spark, Terraform, Data Warehouse, Big Data, Python, Analytics Engineering, Lakehouse Architecture
Click on Apply to know more.
