IAssess Consultants LLP
Website:
iassess.in
Job details:
Profile - Databricks Developer
Experience- 5+ years
Location- Bangalore (On site)
Job Description: -
- Design, build, and optimize data pipelines and ETL/ELT workflows using Databricks and Apache Spark (PySpark).
- Develop scalable, high performance data solutions using Spark distributed processing.
- Lead engineering initiatives focused on automation, performance tuning, and platform modernization.
- Implement and manage CI/CD pipelines using Git-based workflows and tools such as GitHub Actions or Jenkins.
- Collaborate with cross-functional teams to translate business needs into technical solutions.
- Ensure data quality, governance, and security across all processes.
- Troubleshoot and optimize Spark jobs, Databricks clusters, and workflows.
- Participate in code reviews and develop reusable engineering frameworks.
- Should have knowledge of utilizing AI tools to improve productivity and support daily engineering activities.
- Strong knowledge and hands-on experience in Databricks Genie, including prompt engineering, workspace usage, and automation.
Required Skills & Experience
- 5+ years of experience in Data Engineering or related fields.
- Strong hands-on expertise in Databricks (notebooks, Delta Lake, job orchestration).
- Deep knowledge of Apache Spark (PySpark, Spark SQL, optimization techniques).
- Strong proficiency in Python for data processing, automation, and framework development.
- Strong proficiency in SQL, including complex queries, performance tuning, and analytical functions.
- Strong knowledge of Databricks Genie and leveraging it for engineering workflows.
- Strong experience with CI/CD and Git-based development workflows.
- Proficiency in data modeling and ETL/ELT pipeline design.
- Experience with automation frameworks and scheduling tools.
- Solid understanding of distributed systems and big data concepts
Skills: git,ci/cd,data engineering,sql,databricks,apache spark,python
Click on Apply to know more.