JRD Systems
Website:
jrdsi.com
Job details:
Job Title – Databricks/ Pyspark Developer
Location: Hyderabad/Bangalore(Hybrid)
Job Type : Fulltime
** Immediate joiners **
Job Description:
Looking for an offshore Senior Developer (for PMA and supporting EID/CJP) who has experience in Databricks/PySpark, is willing to learn new technologies if needed and is able to work with team. The developer will be mainly focusing on implementing the Delete Act changes needed for PMA . Along with that he will support the optimizations/enhancement for EID/CJP.
Essential Job Functions:
- Design and development of data ingestion pipelines (Databricks background preferred).
- Performance tune and optimize the databricks jobs.
- Evaluates new features and refractors existing code.
- Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures.
- Collaborate with Business Analysts, Technical Manager, Architects and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments).
- Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
- Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary.
- Works on best practices and documenting the process code merges and releases (Bitbucket).
- Works with architect and manager on designs and best practices. Responsible for design of application considering the cost and best practices.
- Good data analysis skills.
- Must be willing to self learn new technologies, become SMEs and develop high quality code in a fast paced environment.
- Mentor junior developers and be hands on in development work.
- Work with QA and automation team. Must have attention to detail to cover all scenarios for testing.
- Handles and supports current production applications.
- Performs code reviews and discussion on the changes with Technology Manager.
Minimum Qualifications and Job Requirements:
- Must be a team player.
- Must have at least 5 years of IT development experience.
- Must have strong analytical and problem-solving skills.
- Must have experience in designing solutions, performing code reviews, mentoring junior engineers.
- Must have strong SQL and backend experience, and working on data driven projects.
- Must have following experience:
- Python/PySpark
- SQL
- Databricks
- SCALA
- SQL
- Spark/Spark Streaming
- Big Data Tool Set
- Linux
- Kafka
Preferred to have skills:
- Azure Data Factory
- Experience collaborating with dev team, project managers, and engineers.
- Excellent communication and teamwork skills
Other Responsibilities:
- Safeguard the company’s assets.
- Adhere to the company’s compliance program.
- Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices.
- Maintain a focus on customer-service, efficiency, quality, and growth.
- Collaborating with additional team members.
- Other duties as assigned.
Click on Apply to know more.