Team Lead - Data Engineer (Python, PySpark ,SQL)
Mopid
- Location
- Bengaluru, Karnataka, India
- Job type
- Full-time
Required skills
- Python
- Agile
- big data technologies
- communication skills
- data engineer
- data mining
- database
- design patterns
- Elasticsearch
- Hadoop
- HBase
- MapReduce
- SQL
About the role
Mopid
Website:
mopid.me
Job details:
Role: Team Lead – Data Engineer (Python, PySpark, SQL)
Experience: 10–17 Years
Responsibilities: - Align with key client initiatives and interface daily with customers across leading Fortune 500 companies to understand strategic requirements
- Ability to understand business requirements and tie them to data engineering solutions
- Open to work from client location as per the demand of the project / customer
- Facilitate technical aspects and develop highly scalable and fault-tolerant distributed data pipelines using Python and PySpark
- Excellent experience in data application development, support, integration development, and quality assurance
- Provide technical leadership and manage the team on a day-to-day basis
- Stay up-to-date on the latest technology to ensure the greatest ROI for customers
- Hands-on coder with good understanding of enterprise-level code
- Design and implement data pipelines, APIs, abstractions, and integration patterns to solve distributed data processing problems
- Experience in defining technical requirements, data extraction, data transformation, automating jobs, productionizing pipelines, and exploring new big data technologies within a parallel processing environment
Qualifications: - 10–17 years track record of relevant work experience and a Computer Science or related technical discipline is required
- Experience in development of enterprise-scale data applications; capable of developing frameworks and design patterns
- Experience with functional and object-oriented programming; Python is a must
- Strong hands-on experience in PySpark and SQL
- Hands-on knowledge in Hadoop ecosystem, MapReduce, HBase, and Elasticsearch
- Development and support experience in Big Data domain
- Experience with database modeling, data mining, and data warehousing
- Experience with data extraction, transformation, and large-scale data processing
- Unit, Integration, and User Acceptance Testing
- Effective communication skills (both written and verbal)
- Ability to collaborate with a diverse set of engineers, data scientists, and product managers
- Comfort in a fast-paced environment
Preferred Qualification: - Experience in Agile methodology
- Proficient with SQL and its variations across popular databases
- Experience working with large, complex datasets from multiple sources
Culture: - Must be a strategic thinker with the ability to think unconventional / out-of-the-box
- Analytical and solution-driven orientation
- Raw intellect, talent, and energy are critical
- Entrepreneurial and Agile mindset; understands the demands of a high-growth environment
- Ability to be both a leader and a hands-on “doer”
Python,Pyspark,Big Data Technologies
Click on Apply to know more.
This page is fully interactive when JavaScript is enabled. Please enable JavaScript to apply or browse related roles.