Egen
Website:
egen.ai
Job details:
We are seeking a Principal Data Architect to define and drive enterprise data architecture, modernization, and governance strategies across complex multi-cloud (Google Cloud Platform and Microsoft Azure) and hybrid environments. This role will design scalable, secure, and high-performing data platforms while partnering with business and engineering leaders to translate strategic objectives into impactful data solutions.
A key focus of this role will be spearheading scalable, agent-driven data migration pipelines to transition legacy on-premises data assets to the cloud, setting the technical direction for engineering teams, and actively managing the customer's technical backlog and delivery roadmap.
Key Responsibilities
- Multi-Cloud Lakehouse Architecture: Lead the end-to-end architecture, design, and implementation of scalable Data Lakehouse solutions across Google Cloud Platform (BigQuery, GCS, BigLake, Dataplex) and Microsoft Azure (Azure Synapse Analytics, Microsoft Fabric, ADLS Gen2, Azure Databricks).
- On-Prem to Cloud Migration & Agentic Pipelines: Architect and deploy automated, scalable, agent-driven data pipelines (leveraging GenAI agents and metadata-driven frameworks) to orchestrate, validate, and accelerate complex migrations from on-premises legacy warehouses/databases (Oracle, Teradata, SQL Server) to GCP and Azure.
- Client Engagement & Advisory: Collaborate directly with customers to understand business goals, data challenges, and technical requirements; translate them into robust architectural blueprints, technical roadmaps, and actionable delivery plans.
- Customer Technical Backlog Management: Own and actively manage the customer technical backlog—translating business epics into technical user stories, prioritizing technical debt, defining sprint milestones, and aligning backlog items with business ROI.
- Modern Ingestion & Streaming: Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration (Cloud Composer, Azure Data Factory, Airflow) and streaming/messaging frameworks (Kafka, GCP Pub/Sub, Azure Event Hubs).
- Governance & Lineage: Establish and enforce best practices for unified data cataloging, metadata management, automated lineage, and data quality across multi-cloud and on-prem systems using Dataplex, Microsoft Purview, or open-source governance tools.
- Security & Compliance: Define and implement robust data security, identity management (IAM / Azure Entra ID), encryption, row/column-level access control, and regulatory compliance standards across clouds.
- Team Technical Leadership & Mentorship: Steer the overarching technical direction of engineering teams; establish architectural standards, mentor senior and mid-level engineers, conduct architecture review, and ensure engineering excellence.
- Hands-on Delivery & Prototyping: Balance strategic architecture discussions with hands-on solutioning, architectural spikes, Proof-of-Concepts (POCs), and performance tuning of complex multi-cloud distributed workloads.
- Cross-Functional Partnership: Partner with enterprise architects, delivery leads, DevOps engineers, and customer stakeholders to drive solution adoption, operational scalability, and long-term platform maintainability
Technical Experience & Tooling
Candidates should demonstrate hands-on architecture and design experience across the following domains:
- Cloud Platforms: Deep expertise in both Google Cloud Platform (GCP) and Microsoft Azure enterprise data ecosystems.
- Processing Engines: Apache Spark, PySpark, Databricks, Azure Synapse / Fabric Spark, GCP Dataproc.
- Modern Migration & AI Automation: Building automated, agentic, or metadata-driven data migration pipelines, automated schema conversion, and data validation agents for on-prem to cloud lift-and-shift/re-platforming.
- Messaging & Streaming: Apache Kafka, GCP Pub/Sub, Azure Event Hubs.
- Orchestration: Apache Airflow, Google Cloud Composer, Azure Data Factory (ADF).
- Analytical Storage & Warehouses: GCP BigQuery, Azure Synapse, Microsoft Fabric, Delta Lake/Parquet.
- NoSQL / Cache Storage: MongoDB, Redis, Bigtable, Azure Cosmos DB.
- Containerization & Deployment: Kubernetes (GKE and AKS), Cloud Run, Azure Container Apps.
- API Frameworks: Python (FastAPI, Flask) for data service endpoints.
- Databases & Querying: Advanced SQL validation and extraction from legacy/RDBMS systems (SQL Server, Oracle, PostgreSQL).
- CI/CD & Version Control: Git, GitHub, GitLab CI/CD, Azure DevOps, and Infrastructure as Code (Terraform).
Required Qualifications & Skills
- 15+ years of progressive experience in Software Engineering, Data Engineering, and Enterprise Data Platform architecture.
- 5+ years architecting, migrating, and deploying production data platforms across GCP and/or Azure, with proven multi-cloud integration patterns.
- Demonstrated track record leading on-premises/cloud to cloud modernization/migration initiatives, including data ingestion, transformation, automated data reconciliation, and cutover strategies.
- Experience designing or integrating autonomous, agent-driven, or LLM-augmented data workflows (e.g., automated schema mapping, code conversion, data quality validation agents).
- Proven hands-on experience designing and deploying Lakehouse platforms incorporating Medallion Architecture (Bronze/Silver/Gold) and Data Mesh / Data Product principles.
- Deep knowledge of multi-cloud data governance, access management, and cataloging (Dataplex, Microsoft Purview, Unity Catalog).
- Strong leadership capability in technical backlog management, agile scoping, sprint planning, and setting technical strategy for distributed teams.
- Exceptional stakeholder communication skills: capability to articulate architectural trade-offs to C-level executives as smoothly as reviewing code with engineers.
Preferred Qualifications (Optional / Good To Have)
- Multi-cloud certifications: Google Cloud Professional Data Engineer / Cloud Architect and/or Microsoft Certified: Azure Data Engineer Associate / Azure Solutions Architect Expert.
- Hands-on knowledge of Snowflake and Databricks
- Familiarity with AWS data services (Amazon S3, Redshift, Glue, EMR).
- Experience with LangChain, LlamaIndex, or agentic framework integrations in data operations.
Education
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or equivalent practical experience.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Click on Apply to know more.
