Website:
arigohrsolution.com
Job details:
🚀 Hiring: Lead Data Engineer – Databricks / PySpark
📍 Location: Hinjewadi, Pune
💼 Experience: 8–14 years
⏳ Notice Period: Immediate to Serving
🏢 Work Model: To be discussed
We are hiring a Lead Data Engineer for a leading AI and technology services organization with a 15,000+ strong workforce across the Asia-Pacific region. The organization delivers AI, digital transformation, cloud, infrastructure, cybersecurity, and technology solutions to enterprise clients across multiple industries.
🔹 Key Responsibilities
- Design, build, and operate scalable data pipelines using Databricks
- Develop end-to-end data workflows from ingestion through transformation to consumption
- Build and optimize PySpark pipelines and Spark jobs
- Modernize legacy ETL processes and refactor code to PySpark
- Work with Delta Lake, Databricks Jobs, Workflows, and notebooks
- Implement data quality, monitoring, validation, and error-handling frameworks
- Troubleshoot production data pipeline issues
- Follow best practices around Git, testing, CI/CD, and code reviews
- Collaborate with architects, analysts, business stakeholders, Infrastructure, Applications, and Cyber teams
- Mentor junior data engineers and provide technical guidance
🔹 Must-Have Skills
✅ 8–14 years of experience in Data Engineering or related roles
✅ 4–8 years of hands-on Databricks experience
✅ Strong PySpark, Python, and SQL skills
✅ Experience building production-scale data pipelines
✅ Strong understanding of ETL/ELT and data engineering principles
✅ Experience with Delta Lake and lakehouse/data modeling concepts
✅ Minimum 2–3 years of team handling/management experience
✅ Strong stakeholder management and mentoring experience
✅ Experience modernizing/refactoring legacy data pipelines
✅ Good understanding of cloud platforms – Azure, AWS, or GCP
✅ Mandatory: Databricks Certified Data Engineer Associate or Professional certification
✅ Job stability – minimum 2 years with an organization preferred
✅ Immediate to serving notice period
🔹 Good to Have
- Databricks Associate Developer for Apache Spark certification
- Azure/AWS/GCP data engineering certifications
- Structured Streaming
- DevOps and CI/CD
- Data quality and testing frameworks
- Data governance and security
📝 Interview Process
2 Technical Rounds + 1 Client Round + HR
If you are an experienced Data Engineer / Lead Data Engineer with strong Databricks + PySpark + Python + SQL expertise and team leadership experience, I'd be happy to connect.
📩 Interested candidates can DM me or share their updated resume at vinay.kumar@arigohrsolution.com
Click on Apply to know more.
