Big Data- Pyspark, Spark, SQL - Pan India Location

Job Summary

Exp – 6- 10 Yrs Location: Pan India Hybrid – Work from Office – 3 Days a week/ 12 days a month Key Skills – Pyspark, Spark, Python, Hive, SQL Job Descriptions: -  6+ Year experience as Data Engineer with experience in building ETL/Data pipelines with cloud technologies -  4+ years extensive experience in designing, developing, and implementing scalable data pipelines using Databricks to ingest, transform, and store structured and unstructured data. -  4+ years' experience programming languages such as Python/Pyspark and query languages like SQL -  4+ years' experience in building metadata driven data ingestion pipelines using ADF -  4+ years' experience in analysing, optimizing, and tuning existing data pipelines for performance, reliability, and efficiency -  2+ years' experience in implementing ML Ops practices to streamline the deployment and management of machine learning models. -  2+ years' experience in utilizing Apache Airflow for job orchestration and workflow management -  Familiarity with CI/CD tools and practices for automating the deployment of data engineering solutions. -  Experience in collaborating with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions -  Knowledge/Experience in implementing security measures and standard processes to ensure data privacy and compliance with regulatory standards -  In-depth knowledge of data engineering concepts, ETL processes, and data architecture principles.