zettamine labs pvt. ltd.
Website:
zettamine.com
Job details:
Hello All,
Greetings from ZettaMine!!!
Job Title: Senior Data Engineer – Databricks
Location: Hyderabad
Experience: 5 - 8 Years
Notice Period: Immediate Joiners
Detailed Job Description
About the Role
As a Data Asset Developer, you will play a critical role in designing robust and scalable data architectures that power business intelligence and analytics across the cybersecurity landscape. You will develop data models that meet the reporting and analytical needs of the cybersecurity team while working closely with Data Scientists and Analytics/BI Engineers to optimize the use of these data models and ensure they are efficient, scalable, and fit for purpose.
You will collaborate with data consumers to understand business requirements, design and deliver high-quality data models, implement data governance best practices, and help establish standards for data pipelines and solutions to ensure seamless data integration and accessibility.
Principal Accountabilities
Data Architecture & Modelling
- Design and implement high-quality, scalable data models that ensure the integrity, security, and accessibility of cybersecurity data assets.
- Collaborate with Analytics/BI Engineers to ensure data models are effectively utilized.
- Produce high-quality technical documentation and metadata.
- Operate data governance processes to publish and manage data assets in accordance with organizational data controls.
- Design and implement data models using industry-standard approaches, including Star and Snowflake schemas.
Databricks Expertise
- Design and implement data architectures using Databricks Notebooks and Unity Catalog.
- Build robust and scalable data pipelines using PySpark, Apache Spark, and SparkSQL.
- Work with large-scale data processing environments and optimize pipelines for performance, scalability, and cost efficiency.
- Develop and maintain data assets supporting reporting, analytics, and business intelligence requirements.
Required Skills & Qualifications
- Overall 5+ years of proven experience in data architecture, data modelling, and database design, including Star and Snowflake schemas.
- Minimum 3 years of hands-on experience with Databricks Notebooks, Unity Catalog, and scalable data pipeline development using PySpark/Spark and SQL.
- Strong hands-on experience working with multi-terabyte datasets, with a focus on performance optimization and cost efficiency.
- In-depth understanding of ETL processes and data warehouse architectures.
- Strong programming experience in Python/PySpark for data engineering and transformation.
- Strong expertise in Apache Spark and SparkSQL.
- Familiarity with the Microsoft Azure cloud platform and modern cloud data solutions.
- Experience with data governance, security, compliance, and data management practices.
- Excellent communication, problem-solving, analytical, and collaboration skills.
- Ability to work effectively with Data Scientists, BI/Analytics Engineers, Data Consumers, and Cybersecurity teams.
Preferred Qualifications
- Experience with Data Lakehouse architectures and modern data engineering practices.
- Knowledge of Machine Learning pipelines and advanced analytics using Databricks.
- Experience with cloud-based data platforms and distributed data processing.
- B.Tech, MCA, M.Tech, or equivalent degree in Computer Science, Information Technology, Engineering, or a related field is preferred.
Interested candidates share your updated CV to praneeth.n@zettamine.com or WhatsApp to 8977318915.
Thanks& Regards,
Praneeth.N
Click on Apply to know more.
