About the role
Role OverviewAs a Python Data Engineer, you will be responsible for processing, analyzing, and optimizing large datasets, building ETL pipelines, and integrating AI models (OCR, NLP, ML-based solutions) into real-world applications. You will work closely with our API developers and product teams to create data-driven features that enhance our platform's capabilities.
Key Responsibilities
Develop and maintain data pipelines, ETL workflows, and automation scripts.
Work with OCR, NLP, and ML models to process text and image data efficiently.
Optimize large datasets for faster retrieval, storage, and processing.
Handle web scraping tasks using Python (BeautifulSoup, Scrapy, Selenium).
Collaborate with API developers to ensure smooth data integration.
Work with cloud-based data storage solutions (AWS, GCP, or Azure).
Improve data quality, validation, and error handling mechanisms.
Required Skills & Experience
Proficiency in Python (Pandas, NumPy, PyTorch/TensorFlow, OpenCV).
Experience in data processing and ETL pipelines (Airflow, Prefect, or custom scripts).
Strong knowledge of databases (SQL & NoSQL – PostgreSQL, MongoDB, MySQL).
Experience in web scraping and data extraction (BeautifulSoup, Scrapy, Selenium).
Familiarity with API integration (RESTful APIs, GraphQL, JSON handling).
Understanding of machine learning or AI model deployment (OCR, NLP, or custom ML models).
Ability to handle large-scale datasets efficiently.
Strong problem-solving skills with a focus on performance optimization.
Bonus Skills (Nice to Have)
Experience with cloud platforms like AWS, GCP, or Azure.
Knowledge of FastAPI, Flask, or Django for data processing APIs.
Familiarity with containerization (Docker, Kubernetes).
Exposure to Hugging Face, OpenAI, or custom AI model integration.
