Innova ESI
Website:
innovaesi.com
Job details:
LinkedIn Job Post – Data Engineer
Job Title: Data Engineer
Experience: 4+ Years
Location: India / Hybrid (as per client requirement)
Job Summary
We are looking for a skilled Data Engineer with strong experience in building and operationalizing large-scale data pipelines and ETL workflows. The ideal candidate should have hands-on expertise with PySpark, Apache Airflow, and AWS data services to design, develop, and maintain scalable data platforms and analytics solutions.
Key Responsibilities
- Build scalable ETL/ELT data pipelines using PySpark on distributed processing frameworks.
- Design and orchestrate workflows using Apache Airflow including DAG creation, scheduling, and monitoring.
- Develop data ingestion and transformation jobs using AWS Glue.
- Manage secure and compliant data access using AWS Lake Formation.
- Maintain and optimize AWS Glue Data Catalog for schema and metadata management.
- Collaborate with analytics teams to publish datasets for BI and dashboards.
- Build and support data visualizations using Amazon QuickSight.
- Ensure data quality, reliability, and performance across all data pipelines.
Required Skills
- Strong hands-on experience with PySpark for large-scale data processing
- Deep knowledge of Apache Airflow DAGs and workflow orchestration
- Expertise in AWS Glue (ETL jobs, Glue Studio, Crawlers)
- Experience with AWS Lake Formation for data governance
- Familiarity with AWS Glue Data Catalog and metadata management
Nice to Have
- Experience with AWS S3, Athena, Redshift, or EMR
- Knowledge of Python-based automation and testing
- Exposure to DevOps / Infrastructure as Code (Terraform or CloudFormation)
Top 5 Mandatory Skills (For Naukri / LinkedIn Search)
- PySpark
- Apache Airflow
- AWS Glue
- AWS Lake Formation
- AWS Glue Data Catalog
Click on Apply to know more.