Website:
yminds.ai
Job details:
Our client is seeking a Data Engineer with 5–8 years of experience in designing and developing scalable data solutions. The ideal candidate will have strong hands-on expertise in Python, PySpark, SQL, and AWS, along with practical experience in Generative AI and Large Language Models (LLMs).
This role requires a solid understanding of data engineering concepts, cloud-based data platforms, and modern data-processing workflows. The candidate should be comfortable working across Agile and Waterfall environments while collaborating effectively with technical and business stakeholders.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data-processing workflows.
- Build data engineering solutions using Python, PySpark, and SQL.
- Develop and optimize SQL scripts for data transformation, validation, and analysis.
- Process and manage large volumes of structured and unstructured data.
- Design and implement cloud-based data solutions, preferably on AWS.
- Integrate data pipelines with Generative AI and LLM-based applications.
- Ensure data quality, reliability, performance, and scalability across data workflows.
- Troubleshoot data pipeline issues and optimize processing performance.
- Collaborate with AI, engineering, product, and business teams to understand data requirements.
- Participate in development activities following Agile and/or Waterfall methodologies.
- Document data pipelines, technical designs, and operational processes.
Required Skills
- 5–8 years of hands-on experience in Data Engineering.
- Strong hands-on experience with Python and PySpark.
- Strong knowledge and practical experience in SQL scripting.
- Solid understanding and hands-on experience with core data engineering concepts.
- Strong hands-on experience with a cloud platform, preferably AWS.
- Experience working in Agile and/or Waterfall development environments.
- Excellent communication, collaboration, and interpersonal skills
Nice-to-Have Skills
- Experience building data pipelines for AI, machine learning, or LLM-powered applications.
- Exposure to AWS data services such as S3, Glue, EMR, Redshift, Lambda, or Athena.
- Knowledge of data lakes, data warehouses, and distributed data-processing architectures.
- Experience with ETL/ELT frameworks and workflow-orchestration tools.
- Familiarity with data modelling, governance, security, and quality practices.
- Exposure to RAG pipelines, vector databases, embeddings, or AI agents.
- Experience with CI/CD pipelines, Git, Docker, or infrastructure automation
About YMinds.AI
YMinds.AI is a technology-focused talent solutions company helping organizations hire exceptional professionals across Engineering, AI/ML, Data, Cloud, Product, and Business functions.
Keywords
Data Engineer, AI Data Engineer, Python, PySpark, SQL, SQL Scripting, AWS, Cloud Data Engineering, Data Pipelines, ETL, ELT, Big Data, Distributed Data Processing, Generative AI, Large Language Models, LLM, RAG, Vector Databases, Data Warehousing, Data Lakes, Agile, Waterfall
Hashtags
#DataEngineer #AIDataEngineer #DataEngineering #Python #PySpark #SQL #AWS #CloudComputing #GenerativeAI #LLM #BigData #DataPipelines #ETL #Hiring #TechJobs #YMindsAI
Click on Apply to know more.
