Website:
trizen-ai.com
Job details:
Airflow Architect — Infrastructure Automation & Orchestration
Location: Chennai
Experience: 8–10 Years
Environment: On-Premise Data Centre & DR
About the Role
We are looking for an experienced Airflow Architect to design, build and own an enterprise-grade Apache Airflow platform across on-premise Data Centre and DR environments.
The primary focus is infrastructure and operations automation, with data pipelines as a secondary workload.
You will be the technical authority for Airflow, owning architecture, HA/DR, security, performance, upgrades and the engineering team operating the platform.
Qualifications
Bachelor’s or Master’s degree in Computer Science, Engineering or equivalent.
Key Responsibilities
Architecture & Platform
- Design and implement highly available Airflow on-premise.
- Architect multi-scheduler, redundant web/API, Celery worker and RabbitMQ infrastructure.
- Design and test DC/DR architecture, including RTO/RPO, failover and failback.
- Establish Dev/UAT/Pre-Prod/Prod/DR environments.
- Own Terraform/Ansible-based Infrastructure as Code and CI/CD for DAG deployment.
- Integrate with LDAP/AD/SSO, Vault/CyberArk, PKI and enterprise storage/logging.
Infrastructure Automation
Build reusable Airflow workflows for:
- OS/middleware patching and rollback
- VM/server/container provisioning
- Backup and restore validation
- Certificate and credential rotation
- Health checks and auto-remediation
- Compliance and configuration-drift checks
- DR drills and evidence collection
- Access provisioning and recertification
- Cross-system runbook automation
Develop custom Operators, Hooks, Sensors and Plugins integrating with Ansible, SSH/WinRM, REST APIs, network, storage, backup and monitoring systems.
Implement safe-automation controls such as approval gates, dry-runs, blast-radius limits, maintenance windows and audit trails.
MySQL Metadata Layer
- Architect and operate MySQL 8.x for Airflow.
- Experience with InnoDB, GTID, replication, InnoDB Cluster/Group Replication and PITR.
- Tune connection pools, InnoDB, queries and Airflow metadata tables.
- Design backup, restore, failover and schema-migration strategies.
Performance & Reliability
- Own Airflow capacity planning and performance tuning.
- Optimise schedulers, workers, concurrency, pools and queue performance.
- Implement monitoring using Prometheus/StatsD, Grafana, ELK/Loki.
- Monitor scheduler health, task latency, queue depth, DB connections and replication lag.
Security & Operations
- Own Airflow, Python dependency, OS, MySQL and RabbitMQ vulnerability management.
- Handle CVE assessment, Nessus/SonarQube findings and VAPT remediation.
- Implement TLS, RBAC, Vault-based secrets, network segmentation and privileged-access controls.
- Maintain SOPs, HLD/LLD, runbooks and operational documentation.
Leadership
- Lead a team of ~8 platform/support engineers.
- Conduct architecture and code reviews.
- Own production SLAs, incidents, changes and technical roadmap.
- Represent the platform in architecture and change-management forums.
Must Have Skills
- 8–10 years Infrastructure / Platform / DevOps / SRE experience.
- 5+ years production Apache Airflow experience.
- Proven experience building and operating Airflow on-premise, including HA and DR.
- Strong MySQL 8.x administration and performance tuning.
- Production experience with Celery Executor and RabbitMQ.
- Strong Python, Linux and shell scripting.
- Hands-on Ansible/AWX and Terraform.
- CI/CD experience with Jenkins, GitLab CI or equivalent.
- Experience with Grafana, Prometheus/StatsD and ELK/Loki.
- Strong understanding of enterprise security, vulnerability management and ITIL processes.
- Proven technical leadership and production SLA ownership.
Good to Have
- Control-M / Autosys / Tidal / Rundeck migration experience.
- ServiceNow / CMDB integration.
- VMware, Veeam, NetBackup or Commvault.
- Vault, Keycloak, LDAP/Kerberos.
- Apache Airflow open-source contributions.
- CKA/CKAD, RHCE, Ansible, MySQL DBA or security certifications.
- Astronomer Apache Airflow 3 certifications.
- BFSI, Telecom or Healthcare experience.
Click on Apply to know more.
_1548133325694.png){kind=logo}