About the Role:
We are currently seeking a Senior Data Engineer to join our team to develop and maintain scalable data pipelines to support growing data volume and complexity. This position involves close interaction with cross-functional teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
Responsibilities:
Data Pipeline Development and Maintenance:
• Design, build, and maintain data pipelines and ETL processes on AWS to support analytics and machine learning initiatives.
• Use AWS services such as RDS, Glue, Lambda, S3 to manage and process large data sets.
DevOps and Automation
• Incorporate DevOps best practices into data engineering workflows, focusing on automation, CI/CD, and infrastructure as code (IaC) for repeatable, scalable solutions.
• Develop and maintain IaC templates (e.g. Terraform, CloudFormation) to provision and manage AWS resources for data engineering and ML tasks.
• Build monitoring, logging, and alerting systems to ensure data pipeline and model uptime, performance, and data quality.
Qualifications:
• Must hold hands-on experience with ETL process with at least 5+ years of experience.
Knowledge, Skills and Abilities Required:
• Cloud Expertise: Strong experience with AWS, including services like S3, Glue, Lambda, and SageMaker.
• Programming: Advanced proficiency in Python, including experience with data processing libraries (e.g. Pandas) and automation.
• DevOps: Experience with DevOps practices, including CI/CD pipelines, Infrastructure as Code (Terraform, CloudFormation), and monitoring (CloudWatch).
• Data Engineering: Solid understanding of data warehousing, ETL/ELT processes.
• Collaboration: Strong communication skills and experience working in cross-functional Agile teams.
• Excellent analytical and problem-solving skills.
• Effective listening skills to understand the requirements of the business.
• Planning, time management and organisational skills.
• The ability to deliver under pressure and to tight deadlines.
