RapidBrains
Website:
rapidbrains.com
Job details:
Job Description
We are looking for an experienced Data QA Engineer with strong expertise in Python, PySpark, SQL, and Data Pipeline Testing to ensure data accuracy, integrity, and quality across legacy and modern data platforms. The ideal candidate should have hands-on experience in automation-driven data validation, ETL/ELT testing, regression testing, and preferably exposure to Capital Markets / Post-Trade processing.
Experience: 5–7 Years
Location : Pune, Gurgaon
Key Responsibilities
- Perform source-to-target data validation across legacy and cloud-based systems.
- Develop Python/PySpark-based automation frameworks for data validation and regression testing.
- Execute complex SQL queries, stored procedures, reconciliation, and schema validation.
- Validate ETL/ELT pipelines, batch jobs, and post-trade data workflows.
- Identify data anomalies, duplicates, transformation issues, and data-integrity violations.
- Integrate automated testing into CI/CD pipelines using Azure DevOps, Jenkins, or GitLab.
- Collaborate with engineering and business teams to implement data-quality controls.
Mandatory Skills
- 5–7 years of experience in Data QA / Data Engineering QA / Automation Testing.
- Strong hands-on experience in Python, SQL, and PySpark.
- Experience in ETL/ELT Testing, Data Pipeline Testing, and Regression Testing.
- Strong understanding of source-to-target validation and data reconciliation.
- Experience with SQL Server, Oracle, PostgreSQL, or similar databases.
- Knowledge of data quality, schema validation, metadata, data lineage, and business-rule validation.
- Exposure to CI/CD tools such as Azure DevOps, Jenkins, or GitLab.
- Familiarity with batch orchestration tools such as AutoSys, Control-M, or Airflow.
- Experience with JIRA, XRay, and Agile methodologies.
Good to Have
- Capital Markets / Post-Trade domain experience.
- Knowledge of Trade Data, Market Data, Reference Data, Clearing, and Settlement.
- Exposure to Azure, AWS, ADF, Databricks, Spark, Hadoop, and Kafka.
- Experience with AI-assisted tools such as Claude or GitHub Copilot.
Click on Apply to know more.