Weekday (YC W21)
Website:
weekday.works
Job details:
This role is for one of the Weekday's clients
Salary range: Rs 1500000 - Rs 2500000 (ie INR 15 - 25 LPA)
Min Experience: 3 years
Location: Kolkata
JobType: full-time
We are seeking a skilled and motivated Data Engineer with a strong foundation in data engineering and exposure to machine learning workflows. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines while enabling data-driven decision-making and supporting ML model development. You will work closely with data scientists, analysts, and engineering teams to ensure high-quality, reliable, and efficient data systems.
Requirements
Key Responsibilities:
- Design, develop, and maintain robust, scalable, and efficient data pipelines for processing large volumes of structured and unstructured data.
- Build and optimize ETL/ELT workflows to ensure timely and accurate data availability across systems.
- Collaborate with data scientists and ML engineers to prepare datasets for training, validation, and production deployment of machine learning models.
- Implement data validation, cleansing, and monitoring processes to ensure data quality, integrity, and consistency.
- Develop and manage data storage solutions using modern data platforms such as data lakes and data warehouses.
- Optimize data infrastructure for performance, scalability, and cost-efficiency in cloud environments.
- Enable real-time and batch data processing systems using appropriate technologies.
- Assist in deploying and maintaining machine learning pipelines, including feature engineering and model monitoring.
- Ensure data security, governance, and compliance with best practices and organizational standards.
- Document data architecture, workflows, and processes for knowledge sharing and maintainability.
Required Skills & Qualifications:
- 3-5 years of hands-on experience in data engineering or related roles.
- Strong proficiency in programming languages such as Python, Scala, or Java.
- Experience with SQL and NoSQL databases, and strong understanding of data modeling concepts.
- Practical experience with big data technologies like Apache Spark, Hadoop, or similar frameworks.
- Familiarity with cloud platforms such as AWS, Google Cloud, or Azure, including data services (e.g., S3, BigQuery, Redshift, or Azure Data Lake).
- Experience building and maintaining ETL/ELT pipelines using tools like Airflow, dbt, or similar orchestration frameworks.
- Solid understanding of machine learning workflows, including data preprocessing, feature engineering, and model lifecycle support.
- Exposure to ML frameworks such as TensorFlow, PyTorch, or Scikit-learn is a plus.
- Knowledge of streaming technologies like Kafka or Kinesis is desirable.
- Strong problem-solving skills and the ability to work with large, complex datasets.
Preferred Qualifications:
- Experience working in cross-functional teams involving data science and product engineering.
- Understanding of MLOps principles and tools for model deployment and monitoring.
- Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes.
- Experience with version control systems like Git and CI/CD pipelines
Click on Apply to know more.