Senior Data Engineer

EY

Website: ey.com
Job details:

We are looking for a Senior Data Engineer with 8+ years of experience and strong hands-on expertise in Kafka and DBT to design and deliver scalable, real-time data pipelines and analytics-ready data platforms on modern cloud architectures.

Key Responsibilities

Design, build, and support scalable real-time and batch data solutions using Kafka and dbt, ensuring high performance and reliability.

Implement real-time data processing and structured streaming solutions, including combining and enriching datasets during streaming workflows.

Design and maintain data lake and lakehouse architectures, including medallion architecture (Bronze, Silver,Gold) patterns for analytics-ready data.

Develop and manage data pipelines with clear segregation, supporting multiple workloads, environments, and performance requirements.

Define and implement data governance and catalog structures, including Unity Catalog, to ensure secure, wellgoverned data access.

Build and maintain data transformations using dbt, enabling reporting, analytics, and downstream data consumption.

Work with modern data technologies such as Python-based data processing, PySpark, Spark SQL, Kafka (Kafka Python), dbt, and AWS/Redshift.

Apply strong data engineering best practices, including data architecture, data modelling, metadata management, and data quality controls.

Analyze and resolve complex data and pipeline issues, taking end-to-end ownership and driving them to resolution.

Collaborate effectively with cross-functional teams, including architects, analytics, and business

stakeholders, to deliver solutions that drive measurable business value.

Skills and Attributes for Success

8+ years of experience in data engineering and analytics platforms.

Strong hands-on experience with Kafka, dbt, Python, PySpark, and Spark SQL.

Proven experience building real-time and batch data pipelines on modern data platforms.

Solid understanding of data lakes, medallion architecture, and pipeline segregation.

Experience with cloud data warehouses (AWS / Redshift) and modern data processing frameworks.

Strong grasp of data engineering best practices, including data modelling and data quality.

Excellent analytical, problem-solving, and collaboration skills.

To Qualify, You Must Have

Bachelor’s or Master’s degree in Computer Science, IT, Engineering, Data Science, or a related field.

8+ years of industry experience in data engineering and analytics platforms.

Proven production-grade experience building and supporting real-time data solutions using Kafka and dbt.

Strong hands-on experience with Python, PySpark, Spark SQL, and modern cloud data warehouses (AWS / Redshift).

Experience designing and implementing data lakes, medallion architecture, pipeline segregation, and data governance frameworks (e.g., Unity Catalog).

Strong understanding of data engineering best practices, including data architecture, data modelling, and data quality management.

Strong communication and stakeholder management skills, with the ability to explain technical solutions clearly.

Ideally, You'll Also Have

Client-facing and stakeholder management experience.

Experience working in large-scale, multi-environment data platforms.

What We Look For Technically strong, curious, and adaptable professionals who enjoy solving challenging data problems and continuously learning new technologies in a fast-moving environment.

What Working at EY Offers

Opportunities to work on diverse, meaningful, and industry-leading projects.

Coaching, learning programs, and a personalized growth and development plan.

Exposure to a collaborative, interdisciplinary work culture.

Flexibility to manage your work in the way that suits you best.

Supportive colleagues and a global environment for continuous knowledge exchange.

EY | Building a better working world

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.

Click on Apply to know more.