Website:
agilegridsolution.com
Job details:
About The Company
Persistent Systems is a leading global technology services company that specializes in software product development, digital transformation, and technology consulting. Renowned for its innovative approach, Persistent leverages cutting-edge technologies to help organizations across various industries achieve their business objectives. With a strong focus on talent development, diversity, and inclusion, Persistent Systems fosters a collaborative work environment that encourages continuous learning and growth. The company's commitment to delivering high-quality solutions has established it as a trusted partner for clients worldwide, making it an ideal place for professionals seeking to advance their careers in technology.
About The Role
We are seeking a highly skilled Data Quality Engineer (Data QE) to join our dynamic team at Persistent Systems. The ideal candidate will play a pivotal role in validating data pipelines, automating testing processes, and ensuring the integrity and reliability of data across end-to-end ETL/ELT workflows. This position requires a proactive individual with extensive experience in data validation, automation testing, and cloud-based data platforms, particularly Databricks. The successful candidate will collaborate closely with data engineering and analytics teams to uphold data quality standards, troubleshoot issues, and optimize data workflows for seamless operations.
Qualifications
The ideal candidate will possess over six years of experience in data quality engineering or data testing roles, with a strong background in automation and data validation. Proficiency in Python, Pytest, and automation frameworks is essential, along with hands-on experience working within Databricks environments. A solid understanding of SQL for complex data validation and reconciliation is required. Candidates should demonstrate excellent problem-solving skills, attention to detail, and the ability to work collaboratively in a fast-paced environment. A bachelor’s degree in Computer Science, Information Technology, or a related field is typically required, with additional certifications in data management or cloud platforms considered a plus.
Responsibilities
The successful candidate will be responsible for a broad range of tasks aimed at maintaining data quality and integrity across various systems. Key responsibilities include:
- Perform comprehensive Data Quality (DQ) validations on ETL/ELT pipelines, including transformation logic, schema validation, and data reconciliation to ensure accuracy and consistency.
- Execute data ingestion testing across multiple sources such as APIs, databases, files, and streaming systems to verify data flow and correctness.
- Validate data flows on Databricks-based ETL frameworks, utilizing notebooks, jobs, and cluster executions to ensure smooth operations.
- Design, develop, and maintain automation test suites using Python and Pytest, focusing on data validation, DQ rule enforcement, and regression testing to streamline testing processes.
- Integrate automation frameworks into CI/CD pipelines to facilitate continuous testing and deployment cycles.
- Build reusable DQ scripts and utilities to enhance data consistency, stability, and monitoring throughout the data pipeline lifecycle.
- Conduct API testing for data ingestion and consumption layers, validating responses, data structures, and downstream system integration.
- Work with Databricks notebooks, data workflows, and jobs to validate ETL pipelines and troubleshoot issues as they arise.
- Leverage AI-driven tools like Databricks Genie or similar to enhance code quality, debugging, and productivity.
- Collaborate with data engineering teams to troubleshoot, optimize, and improve data workflows, ensuring high data quality standards are maintained.
Benefits
Persistent Systems offers a comprehensive benefits package designed to support the well-being and professional growth of its employees. The benefits include a competitive salary structure, health insurance coverage, and a culture that emphasizes talent development. Employees have access to quarterly growth opportunities, company-sponsored higher education, and certifications to foster continuous learning. The organization promotes a vibrant work environment with employee engagement initiatives such as project parties, flexible work hours, and Long Service awards. Additionally, annual health check-ups and insurance coverage—including group term life, personal accident, and Mediclaim hospitalization for employees and their families—are provided to ensure overall well-being.
Values-Driven, People-Centric & Inclusive Work Environment
Persistent is committed to fostering a diverse, inclusive, and equitable workplace. We actively support hybrid work models and flexible hours to accommodate different lifestyles and needs. Our office facilities are designed to be accessibility-friendly, equipped with ergonomic setups and assistive technologies to support employees with physical disabilities. We encourage applications from all qualified individuals, including those with disabilities, and welcome candidates from diverse backgrounds, genders, and orientations. If you have specific requirements, please inform us during the application process or at any point during your employment to ensure appropriate accommodations are provided.
Let's unleash your full potential at Persistent
Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind. We believe in creating a work environment where everyone can thrive and contribute their best.
Click on Apply to know more.