Responsibilities include:
- Validate ETL logic, business logic, and data quality in Snowflake, Databricks, and other data platforms before code changes are released to production.
- Partner with data engineers to identify potential failure points and proactively help catch issues early.
- Ensure the quality of every release using rigorous, data-driven testing practices.
- Develop automated and reusable tests to improve coverage, development velocity, and reduce regression risk.
- Translate business and technical requirements into test scenarios to validate KPIs, metrics, and business rules.
- Contribute to and enhance the existing test automation framework, with a focus on scalability and maintainability.
- Collaborate closely with Data Analysts, Product Managers, and Engineering teams to ensure accuracy, completeness, and usability of the data.
Required Education, Experience/Skills/Training:
- Strong experience validating data pipelines, ETL processes, and data warehouses in production environments.
- Expert-level SQL skills and hands-on experience working with large datasets (terabytes or more), capable of identifying data anomalies through efficient queries.
- Proficiency with Snowflake, Hive, Databricks, and other modern data platforms.
- Solid Python skills and experience with test automation for data pipelines.
- Familiarity with tools like Airflow and Spark and understanding of CI/CD principles.
- Strong collaboration and communication skills; able to work effectively across cross-functional teams.
- B.S. in Computer Science (or equivalent degree or work experience)
Nice to have:
- Experience with BDD frameworks (e.g., Behave)
- Experience working in AWS or other cloud environments
- Familiarity with open-source data quality tools like Deequ, Great Expectations, or similar custom frameworks
