Databricks Developer

The Databricks Developer will be responsible for designing, developing, and optimizing big data processing solutions using Databricks, Apache Spark, and cloud-based services. The ideal candidate should have strong expertise in Python (PySpark), SQL, ETL processes, and data lake architectures to enable advanced analytics and machine learning workflows. Key Responsibilities: Develop and optimize big data pipelines and ETL workflows using Databricks and Apache Spark. Design and implement data lakes, data warehouses, and real-time streaming solutions. Write efficient PySpark or Scala-based Spark jobs for data transformation and processing. Optimize Spark performance through partitioning, caching, and cluster tuning strategies. Work with structured and unstructured data from various sources. Collaborate with data engineers, data scientists, and BI teams to build analytical solutions. Develop Delta Lake solutions for data versioning, ACID transactions, and governance. Integrate Databricks with cloud platforms (AWS, Azure, GCP) and data storage solutions. Implement data security, governance, and compliance best practices. Monitor, troubleshoot, and optimize job execution and data processing performance. Required Skills & Qualifications: Technical Skills: Strong programming experience in Python (PySpark) or Scala. Hands-on experience with Databricks, Apache Spark, and distributed data processing. Proficiency in SQL and query optimization for big data environments. Experience with cloud data services (Azure Synapse, AWS Redshift, Google BigQuery, Snowflake, etc.). Knowledge of Delta Lake, Lakehouse architecture, and structured streaming. Understanding of data partitioning, performance tuning, and job scheduling in Databricks. Familiarity with CI/CD pipelines, version control (Git), and DevOps practices. Experience with data visualization tools (Power BI, Tableau) is a plus. Preferred Qualifications: Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field. Certifications: Databricks or cloud certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, etc.) are a plus.

We are partnering with a globally recognized IT services provider known for delivering cutting-edge data analytics and cloud solutions. And in seeking a skilled Databricks Developer to join their team and contribute to building scalable big data and analytics solutions using Databricks, Apache Spark, and cloud platforms.