Volga Infotech
Company:
https://www.linkedin.com/company/volga-infotech
Seniority: Mid-Senior level
Industries: Software Development
Job details:
** We are looking for immediate joiners only. Interested candidates can share their updated CV ( at resume@volgainfotech.com ) along with their notice period and current/expected compensation. Kindly provide a brief on your work with AWS, specifically AWS Glue and EMR in your email as well **
Job Title
Senior Data Engineer –Data Engineering
Job Description
As the Technical Lead Cloud & Data Engineering, you will be responsible for handling large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data. You will develop complex data transformation ETLs and work extensively with Python, Py Spark, Apache Spark, as well as cloud services from both AWS and Azure. Additionally, you will demonstrate strong expertise in cloud services related to the data domain and have a solid understanding of On-Prem/Cloud Data warehouse databases. You will also exhibit technical leadership capabilities, leading and delivering projects independently.
Experience
4+ Years
Working hours
1:00 pm to 10:00 pm IST ( Mon - Fri )
Qualification
A Bachelor's Degree in Computer Science, or a similar discipline (or comparable practical experience) is required.
Responsibilities
Handle large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data.
Develop complex data transformation ETLs using Python, Py Spark, and Apache Spark.
Utilize cloud services from both AWS and Azure related to the data domain, demonstrating a strong understanding of their respective ecosystems.
Have a solid understanding of On-Prem/Cloud Data warehouse databases and their architecture.
Provide technical leadership, leading and delivering projects independently while ensuring high-quality results.
Stay updated with emerging trends in data tools, analysis techniques, and data usage, understanding their potential impact.
Understand the concepts and principles of data modelling and create, maintain, and update relevant data models for specific business needs.
Possess knowledge of cloud data engineering tools/components/technologies from both AWS and Azure, such as AWS Glue, EMR, Azure Data Factory, etc.
Have an additional advantage with knowledge of Snowflake, Dataiku, Alteryx, or similar technologies.
Collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand requirements and deliver optimal data engineering solutions.
Ensure the scalability, reliability, and efficiency of data pipelines and processes.
Troubleshoot and resolve issues related to data engineering, performance, and scalability.
Document and maintain clear and comprehensive technical specifications, design documents, and best practices.
Collaborate with team members to complete tasks and achieve project objectives.
Ideal Candidate
Experience – 3+ years
A Bachelor's Degree in Computer Science, or a similar discipline (or comparable practical experience) is required.
Extensive experience in handling large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data.
Strong experience in developing complex data transformation ETLs using Python, PySpark, and Apache Spark.
In-depth understanding of cloud services from both AWS and Azure related to the data domain, such as AWS Glue, EMR, S3, Athena, Azure Data Factory, etc.
Solid understanding of On-Prem/Cloud Data warehouse databases and their architecture.
Demonstrated technical leadership capabilities, with the ability to lead and deliver projects independently.
Up-to-date knowledge of emerging trends in data tools, analysis techniques, and data usage.
Communication skills that are effective in conveying complicated design concepts.
Click on Apply to know more.
