DataE2E Technologies
Website:
datae2e.com
Job details:
Platform Engineer Lead (Azure & Databricks)
Location: 100% Remote
Job Type: Full-time
Role Overview
We are seeking a highly experienced Platform Engineer Lead with deep expertise in cloud architecture, specifically within Microsoft Azure and Databricks ecosystems. This role requires a hands-on technical leader who can design, build, and scale enterprise-grade data and platform solutions while mentoring engineering teams and driving best practices across the organization.
Key Responsibilities
- Lead the design and implementation of scalable, secure, and high-performance cloud-based platforms on Azure.
- Architect and manage Databricks-based data platforms, including data ingestion, transformation, and analytics layers.
- Define and enforce platform engineering standards, governance, and best practices across teams.
- Collaborate with cross-functional stakeholders including Data Engineering, DevOps, Security, and Business teams to align platform capabilities with business objectives.
- Drive cloud-native architecture patterns such as microservices, event-driven systems, and distributed data processing.
- Oversee CI/CD pipelines, infrastructure as code (IaC), and automation using tools like Terraform, ARM templates, or Bicep.
- Ensure platform reliability, scalability, and performance through proactive monitoring, logging, and optimization strategies.
- Mentor and lead a team of engineers, providing technical direction, code reviews, and career guidance.
- Evaluate and integrate new technologies to enhance platform capabilities and efficiency.
- Ensure compliance with security, data governance, and regulatory requirements.
Required Skills & Experience
- 10+ years of overall IT experience with at least 3–5 years in a Lead role.
- Strong expertise in cloud architecture, with a focus on Microsoft Azure services (e.g., Azure Data Factory, ADLS, Synapse, AKS).
- Extensive hands-on experience with Databricks, including Spark optimization, Delta Lake, and performance tuning.
- Proven experience designing and implementing enterprise data platforms and distributed systems.
- Strong understanding of data engineering concepts, ETL/ELT pipelines, and big data processing frameworks.
- Experience with DevOps practices, CI/CD pipelines, and Infrastructure as Code (Terraform, ARM, etc.).
- Proficiency in programming languages such as Python, Scala, or SQL.
- Experience with monitoring and observability tools (e.g., Azure Monitor, Log Analytics).
- Solid understanding of security best practices in cloud environments (IAM, networking, data protection).
Click on Apply to know more.