Website:
bestjobtool.com
Job details:
About The Company
Confidential is a leading organization committed to leveraging innovative technology solutions to drive business growth and operational excellence. With a focus on digital transformation, the company specializes in delivering scalable, secure, and high-performance data solutions tailored to meet diverse client needs across various industries. Emphasizing a culture of continuous learning and collaboration, Confidential invests heavily in cutting-edge technologies and talent development to maintain its competitive edge in the market.
About The Role
We are seeking a highly skilled and motivated Azure Data Engineer Lead to join our dynamic team. The ideal candidate will possess 8-12 years of extensive experience in data engineering, with a strong emphasis on cloud-based data solutions on the Microsoft Azure platform. In this pivotal role, you will be responsible for designing, implementing, and optimizing large-scale data architectures that support business intelligence, analytics, AI, and machine learning initiatives. You will lead a team of talented data engineers, providing technical guidance and mentorship to ensure best practices are followed and project objectives are achieved efficiently.
Qualifications
The successful candidate will have a Bachelor's or Master's degree in Computer Science, Engineering, or a related discipline. A minimum of 8-12 years of professional experience in data engineering is required, with at least 5+ years of hands-on expertise in Microsoft Azure cloud technologies. Candidates should demonstrate proficiency in Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Data Lake, and other relevant Azure services. Strong knowledge of data modeling, ETL/ELT processes, and data pipeline architecture is essential. Experience with big data processing frameworks such as Spark, Hadoop, and Kafka, along with expertise in SQL and NoSQL databases, is highly desirable. Certifications such as Microsoft Azure Data Engineer or Databricks Certified Data Engineer are advantageous. Excellent leadership, communication, and collaboration skills are necessary to succeed in this role.
Responsibilities
The Azure Data Engineer Lead will be responsible for a broad range of duties including:
- Architecting and designing scalable, secure, and high-performing data solutions on Azure, ensuring alignment with organizational goals and technical standards.
- Developing and optimizing end-to-end data pipelines to facilitate data collection, processing, and analysis from multiple sources using Azure Databricks, Azure Data Factory, and related services.
- Managing and maintaining cloud data infrastructure, including Azure Data Lake, Azure Synapse Analytics, Azure SQL Data Warehouse, and Azure Blob Storage, to ensure robustness and scalability.
- Collaborating closely with cross-functional teams, including data architects, business analysts, and developers, to gather requirements, define data models, and ensure seamless data integration across systems.
- Designing and building data pipelines to support AI and machine learning workflows, including feature engineering, model training datasets, and real-time inference pipelines utilizing Azure-native tools.
- Partnering with Data Scientists and ML Engineers to productionize machine learning models by implementing robust data workflows, feature stores, monitoring pipelines, and adhering to MLOps best practices on Azure.
- Developing data architectures that support advanced analytics, AI, and Generative AI workloads, ensuring data quality, governance, lineage, and compliance across both structured and unstructured data sources.
- Monitoring, tuning, and optimizing the performance of data workflows and pipelines to maximize efficiency and reduce latency.
- Implementing data governance, security, and compliance standards to protect sensitive data and ensure regulatory adherence.
- Providing technical leadership and mentorship to junior data engineers, fostering a culture of best practices and continuous improvement.
Benefits
The company offers a comprehensive benefits package designed to support the well-being and professional growth of its employees. This includes competitive salary packages, health insurance, retirement plans, and paid time off. Employees have access to ongoing training and certification opportunities to enhance their skills and stay current with industry trends. The organization promotes a collaborative and inclusive work environment, encouraging innovation and knowledge sharing. Additionally, flexible work arrangements may be available to support work-life balance, along with recognition programs to acknowledge outstanding contributions.
Equal Opportunity
Confidential is an equal opportunity employer. We are committed to fostering an inclusive environment where all employees and applicants are treated with respect and fairness, regardless of race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, or any other protected characteristic. We believe diversity enhances our creativity and innovation, and we actively promote a culture of equity and inclusion in all aspects of our operations.
Click on Apply to know more.