Adastra
Website:
adastracorp.com
Job details:
Job Introduction
We are seeking a skilled Azure Databricks Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience designing and maintaining scalable data pipelines using Azure Databricks and Data Factory, strong technical skills in Python and SQL, and familiarity with machine learning concepts. This role requires a proactive, solution-oriented professional who can collaborate effectively with multicultural teams and contribute to high-quality data solutions.
Job Description
As an Azure Databricks Data Engineer, your responsibilities include:
- Technical Requirements Gathering and Development of Functional Specifications.
- Designing, developing, and maintaining scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services.
- Implementing and optimizing Spark jobs, data transformations, and data processing workflows in Databricks.
- Developing and integrating custom machine learning models using Azure Machine Learning, MLflow, and other relevant libraries.
- Leveraging Azure DevOps and CI/CD best practices to automate the deployment and management of data pipelines and infrastructure.
- Conducting troubleshooting on data models.
- Working with Agile multicultural teams in Asia, the EU, Canada, and the USA.
Profile Requirements
For this position of Azure Databricks Data Engineer, we are looking for someone with:
- (Required) At least 5 years of experience in developing and maintaining data pipelines using Azure Databricks, Azure Data Factory, and Spark.
- (Required) Hands-on experience with Unity Catalog.
- (Required) Fluent English communication and soft skills.
- (Required) Knowledge and experience in CI/CD using tools such as Terraform, ARM, and Bicep Script.
- (Required) Solid technical skills in Python and SQL.
- (Required) Familiarity with machine learning concepts, tools, and libraries (e.g., TensorFlow, PyTorch, Scikit-learn, MLflow).
- (Required) Strong problem-solving, communication, and analytical skills.
- (Optional) Willingness to learn and expand technical skills in other areas.
Click on Apply to know more.
