CloudHire
Website:
cloudhire.ai
Company:
https://www.linkedin.com/company/cloudhire1
Seniority: Entry level
Industries: Technology, Information and Internet
Job details:
Position Overview
Cloudhire is seeking a talented and motivated Python Developer specializing in web scraping to join our dynamic team. As a leading AI-based recruitment platform, we leverage cutting-edge technology to connect businesses with top talent. The ideal candidate will possess a strong background in Python programming and a passion for data extraction and manipulation. You will play a crucial role in enhancing our data acquisition capabilities, ensuring we have access to the most relevant information to drive our recruitment solutions.
Job Description
We are looking for a Python Developer with strong Web Scraping experience to build and maintain scalable scraping systems for collecting job listings and other data from websites, job boards, career pages, and online platforms.
- Responsibilities
- Build and maintain web scrapers and crawlers using Python.
- Scrape data from multiple websites and handle different website structures.
- Work with both static and JavaScript-based websites.
- Use tools such as Scrapy, Playwright, Selenium, BeautifulSoup, and Requests.
- Handle pagination, dynamic content, APIs, sessions, cookies, retries, and rate limiting.
- Build concurrent and scalable scraping workflows.
- Clean, normalize, validate, and deduplicate scraped data.
- Store and process large volumes of scraped data.
- Monitor scrapers and troubleshoot failures when websites change.
- Build supporting Python scripts, APIs, and automation as required.
Requirements
Strong hands-on experience with Python.
2+ years of experience in web scraping/crawling.
Strong experience with Scrapy and/or Playwright/Selenium.
Good understanding of HTTP, REST APIs, HTML, CSS, and JavaScript-rendered websites.
Experience with PostgreSQL/MySQL and Redis.
Understanding of concurrency, asynchronous programming, retries, and rate limiting.
Experience building production-grade scraping systems rather than basic one-off scripts.
Good debugging and problem-solving skills.
Good to Have
Experience with large-scale scraping systems.
Experience with AWS, Docker, Celery/RabbitMQ/SQS.
Data pipeline/ETL experience.
Experience scraping job boards, ATS platforms, and company career websites.
Experience with Elasticsearch/OpenSearch.
Location - Hyderabad (Remote)
Budget - 7 LPA
Click on Apply to know more.