Website:
ltm.com
Job details:
Role description
The Azure Databricks Engineer will be responsible for designing, developing, and implementing large-scale data processing solutions on the Azure Databricks platform. The role involves collaborating with data scientists, analysts, and business stakeholders to deliver robust, scalable, and high-performance analytics solutions that support business objectives.
Key Responsibilities
- Develop and optimise data pipelines using Azure Databricks, Spark, and related technologies.
- Design and implement ETL (Extract, Transform, Load) workflows to ingest data from diverse sources into Azure Data Lake and other storage solutions.
- Collaborate with cross-functional teams to understand data requirements and deliver end-to-end solutions.
- Monitor, troubleshoot, and enhance Databricks jobs for performance, reliability, and scalability.
- Ensure data quality, security, and compliance with organisational and regulatory standards.
- Document processes, technical designs, and best practices for ongoing maintenance and knowledge sharing.
Required Skills and Qualifications
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
- Proven experience working with Azure Databricks and Apache Spark.
- Strong proficiency in Python, Scala, or SQL for data engineering tasks.
- Familiarity with Azure cloud services such as Azure Data Lake, Azure Data Factory, and Azure Synapse Analytics.
- Experience in developing and optimising ETL/ELT pipelines.
- Excellent problem-solving, analytical, and communication skills.
Click on Apply to know more.