ProLegion
Website:
prolegion.com
Job details:
Job title : Senior Software Engineer – Python (LLM Evaluation & Repository Validation)
Employment Type: Contractor Assignment (No medical / paid leave)
Experience Required: 4+ Years
Location: Remote
Start Date: Immediate
Engagement Length: 6 Weeks (5 days working)
About the Role
We are looking for experienced Software Engineers (Tech Lead level) who are familiar with high-quality public GitHub repositories and can contribute to this project. You should have experience working with well-maintained, widely-used repositories with 5000+ stars.
This role involves hands-on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality.
Day-to-Day Responsibilities
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Opportunity to lead a team of junior engineers collaborating on projects.
Required Skills
- Strong experience with Python
- Proficiency with Git, Docker, and basic software pipeline setup
- Ability to understand and navigate complex codebases
- Comfortable running, modifying, and testing real-world projects locally
- Experience contributing to or evaluating open-source projects is a plus
Nice to Have
- Previous participation in LLM research or evaluation projects
- Experience building or testing developer tools or automation agents
Note:
1. Applications will only be accepted through the ATS link provided above. Profiles shared through other means will not be considered.
2. Prolegion does not charge any fee from candidates at any stage. If anyone approaches you for money in exchange for this opportunity, treat it as fraud and report it to us immediately.
Click on Apply to know more.