Website:
nasugroup.com
Job details:
We're looking for a Data Engineer to join the Datastore Migration Factory Team, working on a high-impact transformation project for Goldman Sachs. The team is responsible for migrating enterprise data from an on-premise Data Lake to an AWS-hosted Lakehouse.
⚠️ Strong Data Structures & Algorithms (DSA) skills are mandatory. Candidates should be confident in problem-solving and coding assessments.
Key Responsibilities
1. Data Pipeline Migration
- Refactor and migrate extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment.
- Execute end-to-end migration of datasets while ensuring data integrity.
- Collaborate with data owners and stakeholders to facilitate technical hand-off and sign-off of migrated assets.
2. Consumption Pattern Migration
- Convert and optimize legacy SQL and Apache Spark workloads for Snowflake and Apache Iceberg.
- Analyze existing data consumption patterns and deliver optimized data products.
- Work closely with business stakeholders to validate migration outcomes.
3. Data Reconciliation & Quality
- Build and execute reconciliation frameworks to validate migrated data.
- Ensure functional equivalence between legacy and migrated datasets.
- Partner with internal data platform teams and quickly adapt to new tools and workflows.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related field.
- 6+ years of hands-on Data Engineering or Software Engineering experience.
- Strong proficiency in Data Structures & Algorithms (DSA) – Mandatory.
- Strong coding skills in Python or Java.
- Advanced SQL skills with troubleshooting experience.
- Experience with Software Development Life Cycle (SDLC), CI/CD, and Kubernetes (K8s).
Core Data Engineering Expertise
Candidates should have experience with:
- Temporal Data Modeling (SCD Type 2)
- Schema Evolution & Enforcement (Apache Iceberg)
- Data Partitioning & Clustering
- Normalization vs. Denormalization
- Natural Keys vs. Surrogate Keys
Technical Stack
Programming
Data Technologies
- Apache Spark
- Kafka
- Snowflake
- Apache Iceberg
- Hadoop (HDFS/Hive)
- Sybase IQ
Data Formats
What We're Looking For
- Strong DSA and coding fundamentals (Mandatory)
- Excellent analytical and problem-solving skills
- Ownership mindset with a focus on quality and delivery
- Strong communication and stakeholder management skills
- Ability to collaborate effectively with global teams
- Curiosity to learn and adapt to new technologies
Click on Apply to know more.