Zorba AI
Website:
zorbaconsulting.in
Company:
https://www.linkedin.com/company/zorba-consulting-india
Seniority: Mid-Senior level
Industries: IT Services and IT Consulting
Job details:
We are looking for an experienced
Data Engineer with strong hands-on expertise in
Databricks, PySpark, SQL, and Snowflake to design, develop, optimize, and maintain scalable data engineering solutions. The ideal candidate should have solid experience in building data pipelines, processing large datasets, and working with cloud-based data platforms.
Experience or knowledge of
Databricks Genie and
Snowflake CoCo is preferred. However, candidates who demonstrate strong technical fundamentals, a positive attitude, and willingness to learn these emerging capabilities will also be considered.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
- Develop efficient and optimized SQL queries for data transformation, extraction, and analysis.
- Build and manage ETL/ELT pipelines for processing structured and semi-structured data.
- Work extensively with Snowflake for data ingestion, transformation, storage, and analytical workloads.
- Implement data processing solutions using Apache Spark, PySpark, and Databricks.
- Work with Delta Lake and implement reliable data lakehouse solutions.
- Optimize Spark jobs, SQL queries, and data pipelines for performance and cost efficiency.
- Develop reusable and maintainable Python/PySpark code following engineering best practices.
- Troubleshoot data pipeline failures, performance issues, and production incidents.
- Implement data quality, validation, monitoring, and error-handling mechanisms.
- Collaborate with data architects, analysts, developers, and business stakeholders to understand requirements and deliver data solutions.
- Participate in code reviews, testing, deployment, and production support activities.
- Explore and adopt new Databricks and Snowflake capabilities to improve data engineering productivity.
Required Skills
- 4–8 years of experience in Data Engineering or a related field.
- Strong hands-on experience with Databricks.
- Excellent programming skills in PySpark and Python.
- Strong expertise in SQL, including complex queries, joins, CTEs, window functions, and query optimization.
- Practical experience with Snowflake and cloud data warehousing concepts.
- Experience developing ETL/ELT data pipelines.
- Strong understanding of Apache Spark architecture and distributed data processing.
- Experience with Delta Lake/Lakehouse architecture.
- Good understanding of data warehousing, data modeling, and data integration concepts.
- Strong debugging, problem-solving, and analytical skills.
Preferred Skills
- Knowledge or hands-on exposure to Databricks Genie.
- Knowledge or hands-on exposure to Snowflake CoCo.
- Experience with AWS or Azure cloud platforms.
- Exposure to Git, CI/CD, DevOps, and automated deployment processes.
- Experience with workflow orchestration tools such as Airflow or similar platforms.
- Understanding of data governance, security, and data quality practices.
Qualification
- B.E./B.Tech/MCA/M.Tech in Computer Science, Information Technology, Engineering, or a related discipline.
- Equivalent professional experience with strong technical expertise will also be considered.
Candidate Profile
We value candidates who have strong fundamentals in
Databricks, PySpark, SQL, and Snowflake and are comfortable learning new technologies.
Genie and CoCo experience is desirable but not mandatory for candidates who demonstrate the ability and willingness to learn.
The successful candidate will be a proactive team player with strong communication skills, an ownership mindset, and the ability to work effectively in a fast-paced data engineering environment.
Skills: genie,coco,cloud,databricks,snowflake,sql,pyspark
Click on Apply to know more.