Insight Global
Website:
insightglobal.com
Job details:
Insight Global is seeking a highly experienced Senior Data Engineer to design, build, and optimize data pipelines that integrate metadata and lineage between Snowflake and Collibra. This engineer will play a key role in enabling enterprise-grade data governance by automating metadata ingestion, synchronization, and lineage flows across cloud data platforms and governance tools. This is a hands-on engineering role focused on building metadata pipelines, API-based integrations, and scalable orchestration workflows that ensure Collibra always has accurate, up‑to‑date information from Snowflake.
Must Haves:
- Strong Data Engineering experience
- Expert-level Python for API integration, metadata processing, and automation
- Strong SQL experience with Snowflake (SnowSQL, query optimization, access patterns)
- Experience working with Collibra APIs, import/export workflows, and governance operating models
- Proficiency with REST APIs, JSON, and metadata transformation
- Strong understanding of ETL/ELT patterns, data warehousing concepts, and cloud-native architectures
- Experience with version control, CI/CD, and DevOps tooling (GitHub Actions, Azure DevOps, Jenkins, etc.)
- Familiarity with metadata concepts: business glossary, data domains, assets, schemas, lineage
- Understanding of data quality frameworks and standards
Plusses:
- Experience with Snowflake metadata tables, INFORMATION_SCHEMA, and ACCOUNT_USAGE views
- Hands-on work with Collibra Workflow Designer, Collibra import jobs, or custom integration scripts
- Familiarity with DBT, Databricks, or cloud data engineering (AWS, Azure, GCP)
- Background in highly regulated industries (financial services, energy, government, healthcare)
Click on Apply to know more.