UST
Website:
ussmokeless.com
Company:
https://www.linkedin.com/company/ustglobal
Seniority: Entry level
Industries: IT Services and IT Consulting
Job details:
Candidates ready to join immediately can share their details via email for quick processing.
📌 CCTC | ECTC | Notice Period | Location Preference
nitin.patil@ust.com
Act fast for immediate attention! ⏳📩
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark
- Build and manage batch and real-time data processing systems
- Develop and integrate Kafka-based streaming solutions
- Optimize Spark jobs for performance, cost, and scalability
- Work with cloud-native services to deploy and manage data solutions
- Ensure data quality, reliability, and security across platforms
- Collaborate with data scientists, analysts, and application teams
- Participate in code reviews, design discussions, and production support
Must-Have Skills
- Strong hands-on experience with PySpark / Apache Spark
- Solid understanding of distributed data processing concepts
- Experience with Apache Kafka (producers, consumers, topics, partitions)
- Hands-on experience with any one cloud platform:
- AWS (S3, EMR, Glue, EC2, IAM) or
- Azure (ADLS, Synapse, Databricks) or
- GCP (GCS, Dataproc, BigQuery)
- Proficiency in Python
- Strong experience with SQL and data modeling
- Experience working with large-scale datasets
- Familiarity with Linux/Unix environments
- Understanding of ETL/ELT frameworks
- Experience with CI/CD pipelines for data applications
Good-to-Have Skills
- Experience with Spark Structured Streaming
- Knowledge of Kafka Connect and Kafka Streams
- Exposure to Databricks
- Experience with NoSQL databases (Cassandra, MongoDB, HBase)
- Familiarity with workflow orchestration tools (Airflow, Oozie)
- Knowledge of containerization (Docker, Kubernetes)
- Experience with data lake architectures
- Understanding of security, governance, and compliance in cloud environments
- Exposure to Scala or Java is a plus
- Prior experience in Agile/Scrum environments
Click on Apply to know more.
