Website:
novio.in
Job details:
Job Role: Data Engineer
Location: Mumbai, Andheri.
WFO(Monday-Friday)
Looking for Immediate, 15 days joiner.
Required Skills
- 3+ years of hands-on experience as a Data Engineer.
- Strong proficiency in Python and PySpark programming for data engineering tasks.
- In-depth knowledge of ETL processes and data pipeline architectures.
- Experience with Airflow or Step Functions for orchestration and scheduling.
- Solid experience working with AWS services.
- Proficiency in building and maintaining CI/CD pipelines
- Experience with data modelling, database design, and querying.
- Strong problem-solving skills, with the ability to troubleshoot and optimize complex data pipelines.
Bonus Skills
- Knowledge of cloud platforms (e.g., AWS, GCP).
- Experience with containerization and Kubernetes.
- Experience in data security, encryption, and compliance best practices.
- Strong communication and collaboration skills.
Responsibilities
- Design, build, and maintain scalable and efficient ETL/ELT pipelines to process large-scale datasets.
- Implement and manage workflow automation and orchestration using Apache Airflow or Step Functions.
- Build and optimize data infrastructure using AWS services.
- Integrate and manage data warehouses.
- Design and develop dynamic and interactive dashboards using PowerBI, Metabase, and Superset, to present insights.
- Develop and maintain CI/CD pipelines for seamless deployment and continuous integration of data solutions.
- Monitor and troubleshoot data pipeline issues, ensuring data quality and consistency.
- Leverage best practices for data governance, security, and compliance in cloud environments.
- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver reliable data solutions.
Skills:- Data engineering, Apache Airflow, ETL, CI/CD, Amazon Web Services (AWS) and PySpark
Click on Apply to know more.