Sky Systems, Inc. (SkySys)
Website:
myskysys.com
Job details:
Role: Senior Big Data Developer
Position Type: Full-Time Contract (40hrs/week)
Contract Duration: Long Term
Work Schedule: 8 hours/day (Mon-Fri)
Location: Hybrid 2x Days onsite - (Wed/Thurs) Hyderabad, India
We are seeking an experienced Senior Big Data Developer to design, develop, and support large-scale data processing applications on AWS Cloud. The ideal candidate will have strong experience building distributed data platforms using Apache Spark, Hadoop, Scala, PySpark, Java, and cloud-native AWS services.
This role focuses on developing scalable batch processing pipelines, API-driven services, and cloud-native Big Data solutions that support high-volume enterprise data processing.
Key Responsibilities
- Design, develop, and maintain scalable Big Data applications and distributed data processing pipelines.
- Build high-performance batch processing solutions using Apache Spark, Hadoop, and AWS.
- Develop REST APIs and microservices supporting enterprise data platforms.
- Build cloud-native applications utilizing AWS services including EMR, EC2, ECS, S3, API Gateway, Step Functions, and Airflow.
- Develop applications using Scala, PySpark, Java, Kotlin, and Python.
- Optimize Spark, Hadoop, and distributed workloads for performance and scalability.
- Develop Linux shell scripts and Python automation.
- Collaborate with Data Engineering, Architecture, DevOps, and Product teams.
- Build and maintain CI/CD pipelines using Jenkins and AWS DevOps tools.
- Troubleshoot production issues and participate in code reviews.
Required Qualifications
- 7+ years of experience developing Big Data applications.
- Strong experience building scalable distributed data processing platforms.
- Hands-on experience with Apache Spark, Hadoop, HDFS, MapReduce, and Cassandra.
- Strong programming experience with Scala, PySpark, Java, Kotlin, and Python.
- Experience developing REST APIs and Microservices.
- Strong experience with AWS services including EMR, EC2, ECS, S3, API Gateway, Step Functions, and Airflow.
- Experience with Linux, Shell Scripting, and Python automation.
- Experience building CI/CD pipelines using Jenkins, Maven, Gradle, Artifactory, AWS CodeCommit, and CloudFormation.
- Excellent analytical, troubleshooting, and communication skills.
Click on Apply to know more.