frelance
Website:
jmsalazar.net
Company:
https://www.linkedin.com/company/blogs.allyfu.com
Seniority: Mid-Senior level
Industries: Food & Beverages
Job details:
Senior AWS Data Lake Framework Data Engineer
Experience: 7+ years of relevant experience,
Location: Remote Location, Pan India. Full time work from home,
Shift: UK working hours,
Budget: 20 to 25 LPA
Must Have: Data Lake, AWS, Building pipeline
Notice Period: 30-45 days joiner
---------------------------------------------
Responsibilities:
· Lead migration from MS SQL/RDBMS to AWS Data Lake with data integrity & quality
· Design & build reusable data pipeline frameworks (not one-off pipelines) using Medallion Architecture (Bronze → Silver → Gold)
· Implement data modeling standards (Star Schema, Snowflake Schema, SCD Type 1/2/3)
· Hands-on development & deployment of production pipelines using the framework while mentoring team adoption
· Optimize AWS costs through right-sizing, partitioning, workload-aware architecture & continuous monitoring
· Select & implement optimal tech stack for ingestion, transformation, orchestration & cataloging
· Build with Infrastructure as Code (Terraform/CloudFormation) and establish CI/CD for data pipelines
Requirements:
· Proven RDBMS-to-data lake migration experience (MS SQL/Oracle → AWS)
· Expertise in Medallion Architecture implementation in AWS
· Strong data modeling skills (Star Schema, Snowflake Schema, Data Vault, SCD)
· Framework building experience – reusable, parameterized, metadata-driven pipeline frameworks
· AWS cost optimization mastery – pricing models, Spot/On-Demand, storage tiering, query optimization
· AWS Services Proficiency:
· S3 (lifecycle, tiering)
· AWS Glue/EMR (Spark/PySpark optimization)
· MWAA/Step Functions (orchestration)
· Glue Data Catalog/Lake Formation
· Athena
· Apache Iceberg/Delta Lake/Hudi
· Infrastructure as Code: Terraform/CloudFormation
· CI/CD for data pipelines (GitLab CI/Jenkins/AWS CodePipeline)
· AWS Certifications preferred (Solutions Architect, Data Analytics, DevOps)
Click on Apply to know more.
