Website:
mindbrain.co.in
Job details:
Job Title: Big Data Engineer (Python / PySpark / Snowflake)
Experience: 5+ Years
Employee Location: Gurgaon / Noida
Shift Timing: 2:00 PM – 11:00 PM
Work Location: Remote
Job Summary
We are looking for an experienced Big Data Engineer with strong expertise in Python, PySpark, Spark, SQL, and Snowflake. The candidate will be responsible for developing and managing large-scale data processing pipelines and working with modern big data and cloud technologies.
Key Responsibilities
- Develop and maintain big data pipelines using Python, PySpark, and Apache Spark.
- Work with Spark Streaming, Spark SQL, Hive, and Hadoop for large-scale data processing.
- Design and optimize data models and queries using SQL and Snowflake.
- Integrate and process data using Kafka, HBase, and MongoDB.
- Deploy and manage data workloads on AWS services (S3, EC2, EMR, Lambda, Athena, Kinesis).
- Implement CI/CD pipelines and DevOps practices for data engineering workflows.
- Ensure data quality, performance optimization, and scalability of big data solutions.
Required Skills
- Python, PySpark, Apache Spark
- SQL, Snowflake
- Hadoop, Hive, Spark Streaming
- Kafka, HBase, MongoDB
- AWS (S3, EC2, EMR, Lambda, Athena, Kinesis)
- CI/CD and DevOps practices
Click on Apply to know more.