Website:
mindbrain.co.in
Job details:
Job Title: Big Data Testing Engineer (AWS Data Pipelines)
Experience: 5–6 Years
Location: Remote (India)
Shift: IST
Contract Duration: 6-12 Months (Extendable)
Engagement Type: Staff Augmentation
Job Summary
We are looking for a Big Data Testing Engineer with hands-on experience in testing data pipelines and validating large-scale data processing workflows. The ideal candidate should have strong experience working with AWS data services and big data environments, ensuring data accuracy, integrity, and performance across data pipelines.
Key Responsibilities
- Perform end-to-end testing of Big Data pipelines and ETL workflows.
- Validate data ingestion, transformation, and loading processes across large datasets.
- Work with AWS data services to test and monitor pipeline performance and reliability.
- Execute data validation, reconciliation, and quality checks to ensure data integrity.
- Write and optimize SQL queries for data verification and testing.
- Collaborate with data engineers and developers to troubleshoot data issues and pipeline failures.
- Develop test strategies, test cases, and automation scripts for data testing.
- Perform regression, integration, and performance testing in big data environments.
- Ensure compliance with data quality standards and best practices.
Required Skills
- Strong experience in Big Data Testing and Data Pipeline Testing
- Hands-on experience with AWS Data Services such as S3, Glue, Redshift, Lambda, EMR, Athena
- Good understanding of ETL/ELT workflows and data processing frameworks
- Strong SQL skills for data validation and testing
- Experience with data quality checks and large dataset validation
- Knowledge of test automation tools and frameworks for data testing
- Familiarity with Agile/Scrum development environments
Preferred Skills
- Experience with Apache Spark / Hadoop ecosystem
- Exposure to Python or scripting for test automation
- Knowledge of CI/CD pipelines for data platforms
Click on Apply to know more.