Quess Corp Limited
Website:
quesscorp.com
Company:
https://www.linkedin.com/company/quess-corp
Seniority: Associate
Industries: IT Services and IT Consulting
Job details:
Job Description: Big Data Testing
Position: Big Data Testing Engineer / QA Engineer – Big Data
Experience: 5–7 years (relevant)
About the Role
We are seeking an experienced Big Data Testing Engineer to ensure the quality, accuracy, and reliability of large-scale data pipelines and platforms. The ideal candidate will have strong experience in designing test strategies for Hadoop big data systems, tracking defects systematically, and collaborating effectively with development and business teams.
Key Responsibilities
- Design, develop, and execute test plans and test cases for big data pipelines, ETL processes, and data platforms
- Validate data accuracy, completeness, and consistency across source and target systems
- Perform functional, regression, integration, and data reconciliation testing on large datasets
- Create and maintain detailed test cases, test scripts, and test data covering various data scenarios and edge cases
- Log, track, and manage defects through their lifecycle using Jira, ensuring timely resolution and closure
- Collaborate with developers, business analysts, and data engineers to understand requirements and translate them into test scenarios
- Perform root cause analysis of data quality issues and defects
- Validate data transformations, aggregations, and business logic implemented in Spark/Hadoop/ETL jobs
- Prepare test summary reports, defect reports, and quality metrics for stakeholders
- Participate in sprint planning, daily stand-ups, and retrospectives in Agile environments
- Ensure adherence to testing standards, best practices, and documentation guidelines
Required Skills & Experience
- 5–7 years of overall QA/testing experience, with substantial hands-on experience in Big Data testing
- Strong experience in creating and executing test cases for data pipelines and ETL workflows
- Hands-on experience in defect tracking and management using Jira
- Solid understanding of Big Data technologies (Hadoop, Hive, Spark, HDFS) from a testing/validation perspective
- Experience writing SQL queries for data validation and reconciliation
- Familiarity with test management tools and defect lifecycle processes
- Understanding of data warehousing concepts and ETL testing methodologies
- Experience working in Agile/Scrum environments
- Strong analytical and problem-solving skills with attention to detail
- Excellent verbal and written communication skills — must be able to clearly document findings and communicate effectively with cross-functional teams and stakeholders
Good to Have
- Exposure to automation testing frameworks for data pipelines (e.g., PyTest, Great Expectations)
- Experience with cloud data platforms (AWS/Azure/GCP)
- Familiarity with Databricks or PySpark
- Knowledge of scripting languages (Python/Shell) for test automation
- Experience with CI/CD pipelines and version control (Git)
- Exposure to performance/load testing for big data systems
Soft Skills
- Strong team player with ability to work independently and take ownership
- Ability to manage multiple priorities and meet deadlines under pressure
- Client-facing communication experience is a plus
Click on Apply to know more.
