Senior Data Engineer - GPT

Salary

₹28 - 35 LPA

Min Experience

5 years

Location

Chennai

JobType

full-time

About the role

Hiring for our leading client product based in Chennai (Hybrid) 

About Company:
➞ Join a leading company in the ad-tech space, backed by a Nasdaq-listed parent company with over 5,000+ employees globally.
➞ Specializes in providing data-driven solutions for demand-side platforms (DSP).
➞ Enables global brands to connect with target audiences through precision-driven approaches.
➞ Leverage a real-time analytics platform to optimize advertiser performance across digital platforms.

Qualifications: 

  • 5-8 years of experience in data engineering, with a focus on building and managing data pipelines.
  • Strong proficiency in relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
  • Experience in building data pipelines with data warehouses like SnowflakeRedshift 
  • Experience in processing unstructured data stored from S3 using Athena, Glue etc.
  • Hands-on experience  or knowledge with Kafka for real-time data streaming and messaging.
  • Solid understanding of ETL processes, data integration, and data pipeline optimization.
  • Proficiency in programming languages like PythonJava, or Scala for data processing.
  • Experience with Apache Spark for big data processing and analytics is an advantage
  • Familiarity with AWS cloud platforms for data infrastructure is a plus.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.

 

Key Responsibilities:

  • Design, build, and maintain efficient and scalable data pipelines to support data integration and transformation across various sources.
  • Work with relational databases (e.g., MySQL, PostgreSQL, etc.) and NoSQL databases (e.g., MongoDB, Cassandra, etc.) to manage and optimize large datasets.
  • Utilize Apache Spark for distributed data processing and real-time analytics.
  • Implement and manage Kafka for data streaming and real-time data integration between systems.Collaborate with cross-functional teams to gather and translate business requirements into technical solutions.
  • Monitor and optimize the performance of data pipelines and architectures, ensuring high availability and reliability.
  • Ensure data quality, consistency, and integrity across all systems.
  • Stay up-to-date with the latest trends and best practices in data engineering and big data technologies.
     

Skills

NoSQL
Kafka
PostgreSQL
AWS
GCP
Azure Kubernetes Service - AKS
Data Engineer
ETL
Problem solving