About the role
Job Title: Data Engineer
Job Summary:
Are you passionate about building scalable data pipelines, optimizing ETL processes, and designing efficient data models? We are looking for a Databricks Data Engineer to join our team and play a key role in managing and transforming data in Azure cloud environments.
In this role, you will work with Azure Data Factory (ADF), Databricks, Python, and SQL to develop robust data ingestion and transformation workflows. You'll also be responsible for integrating SAP IS-Auto data, optimizing performance, and ensuring data quality & governance.
If you have strong experience in big data processing, distributed computing (Spark), and data modeling, we'd love to hear from you!
Key Responsibilities:
Develop & Optimize ETL Pipelines:
Build robust and scalable data pipelines using ADF, Databricks, and Python for data ingestion, transformation, and loading.
Data Modeling & Systematic Layer Modeling:
Design logical, physical, and systematic data models for structured and unstructured data.
Integrate SAP IS-Auto:
Extract, transform, and load data from SAP IS-Auto into Azure-based data platforms.
Database Management:
Develop and optimize SQL queries, stored procedures, and indexing strategies to enhance performance.
Big Data Processing:
Work with Azure Databricks for distributed computing, Spark for large-scale processing, and Delta Lake for optimized storage.
Data Quality & Governance:
Implement data validation, lineage tracking, and security measures for high-quality, compliant data.
Collaboration:
Work closely with business analysts, data scientists, and DevOps teams to ensure data availability and usability.
Testing and Debugging:
Write unit tests and perform debugging to ensure the Implementation is robust and error-free.
Conduct performance optimization and security audits.
About the company
techcarrot is a leading global IT service provider and partner in digital transformation offering cost-effective, customer-centric, and cutting-edge digital solutions to clients worldwide.
founded in 2016, techcarrot is headquartered in Dubai, UAE. Being a people-oriented organization with a philosophy of "We care" as our core DNA, techcarrot continually focuses on new initiatives and strategies to increase overall employee experience by providing a plethora of platforms to drive employee development, overall efficiency, increase overall employee satisfaction. With over 200+ technology specialists who are engaged and motivated we are growing at 20% a year-on-year since inception.
We offer cutting-edge services across multiple industries and deliver seamless customer experiences, ensuring high value and rapid growth for our clients. techcarrot has earned wide recognition as a reliable partner for organizations across the globe because of its professional approach and strong delivery capabilities. The Middle East and Africa are handled by UAE operations, while APAC is handled by India operations. We have also expanded in the USA and the Netherlands as well.
