Vtricks Technologies
Website:
vtricks.in
Job details:
Job Title: Senior Data Pipeline Engineer
Location: Chennai / Bangalore / Hyderabad
Experience: 5–8 Years
Employment Type: Full-time
Qualification: Bachelor’s Degree (Computer Science / IT / Data Engineering or related field)
Job Summary
We are looking for an experienced Senior Data Pipeline Engineer to design, develop, and maintain scalable data pipelines for enterprise analytics and AI initiatives. The role involves integrating data from multiple systems such as SAP, SCADA, AMI, and APIs into a centralized analytics platform built on Elasticsearch.
Key Responsibilities
Data Pipeline Development
- Design and build scalable ETL/ELT pipelines for enterprise data ingestion
- Integrate data from systems like SAP, CMS, QMS, AMI/HES, SCADA, and web portals
- Develop robust data transformation logic and data models
Data Integration & Processing
- Build pipelines using Python, SQL, and REST APIs
- Implement data validation, logging, monitoring, and error handling
- Optimize pipelines for performance and scalability
Analytics & KPI Engine
- Develop business logic engines for:
- KPI computation
- SLA calculations
- Operational analytics
- Implement complex transformation logic for dashboarding and reporting
🔍 Elasticsearch Platform
- Design and manage Elasticsearch indices, mappings, and queries
- Optimize indexing and search performance
- Support data lifecycle management and query optimization
🤖 AI / ML Data Enablement
- Build pipelines to support AI/ML model training and deployment
- Ensure high-quality and reliable data availability
Collaboration & Support
- Work with analytics teams, dashboard engineers, and business stakeholders
- Support testing, validation, and troubleshooting
- Maintain and enhance existing data pipelines
Required Skills
- Strong experience in Python for data engineering
- Hands-on experience in ETL/ELT pipeline development
- Strong SQL and data modeling skills
- Experience working with REST APIs and enterprise integrations
- Hands-on experience with Elasticsearch
- Experience integrating with SAP (RFC/BAPI/ODATA/APIs)
- Knowledge of pipeline orchestration and scheduling tools
- Strong debugging and performance optimization skills
Preferred Skills
- Experience with Kibana / Elastic dashboards
- Exposure to AI/ML data pipelines
- Experience with cloud platforms (AWS/Azure/GCP)
- Knowledge of streaming data processing (Kafka, etc.)
- Familiarity with data quality and monitoring frameworks
- Experience in large-scale enterprise data environments
Education
- Bachelor’s degree in Computer Science, IT, Data Engineering, or related field
Soft Skills
- Strong analytical and problem-solving abilities
- Good communication and stakeholder management skills
- Ability to work in a fast-paced enterprise environment
- Strong documentation and collaboration mindset
🚀 Why Join Us?
- Work on enterprise-scale data platforms and pipelines
- Opportunity to contribute to AI-driven analytics initiatives
- Exposure to complex data ecosystems and cutting-edge technologies
Click on Apply to know more.