Skills Agency
Website:
skills-agency.com
Job details:
Company Description
Skills Agency specializes in software development and IT consulting services across various domains, with a core expertise in Digital Marketing. The team is highly adaptable to client requirements and supported by bilingual French-speaking project managers.
Job Summary
We are looking for a highly skilled Python & AI Developer with strong expertise in large-scale web scraping, data extraction, and AI-augmented automation. The ideal candidate will build resilient data pipelines that process massive datasets, validate results in real time, and integrate AI tools to handle ambiguous cases efficiently. This role requires deep Python proficiency, strong backend engineering skills, and the ability to design autonomous systems that can run reliably without manual intervention.
- Scraping & Data Extraction: A large part of the job is retrieving data from very large datasets (like Common Crawl, HTTP Archive, CrUX, Sirene), filtering it using Python (and AI for ambiguous cases), and performing live validation.
- Async/Concurrency: Experience with AsyncIO, aiohttp, or httpx is important. A key part of the job is running concurrent checks (Live Validation) efficiently without crashing the script. This was a weak point previously.
- Data Handling Logic: Capable of writing efficient scripts to parse large text inputs (Regex, JSON streams) and structure them into clean CSV/Parquet files.
- Autonomous Error Handling: Ability to implement robust error handling (retries, logging, try/except blocks) so the pipelines don't stop overnight.
- AI-Augmented Workflow: I am looking for an "AI-First" developer who uses ChatGPT/Copilot to code faster (generating Regex, optimizing snippets), but who understands that AI must be guided/monitored and that the output must be systematically verified and corrected.
Technical Stack:
Python 3.10+, PostgreSQL (basic), Docker, Git. (Playwright/Selenium is a +).
Location : Indiranagar, Bangalore
Click on Apply to know more.