We are seeking a Founding Engineer to develop the core intelligence behind a cutting-edge large language model (LLM) gateway platform. This role involves designing and implementing systems that determine, on a per-request basis, the optimal provider, whether to use real-time or batch processing, caching strategies, and selecting model tiers to achieve the lowest cost while maintaining quality and meeting service level agreements (SLAs).
Responsibilities:
- Build and optimize routing algorithms that decide request handling dynamically.
- Develop evaluation frameworks to assess model and provider performance.
- Design token economics and cost models to ensure efficient resource utilization.
- Collaborate closely with cross-functional teams to align technical solutions with business goals.
- Contribute to the founding engineering team shaping the product architecture and infrastructure.
Requirements:
- 3 to 4 years of relevant engineering experience.
- Strong background in machine learning concepts, particularly routing and evaluation methodologies.
- Experience with real-time and batch processing systems.
- Understanding of economic models related to token usage or resource allocation.
- Ability to work in a fast-paced startup environment and contribute to foundational technology.
Location and Compensation:
- Position is based in Bengaluru, Karnataka, India, with flexible remote work options.
- Competitive salary range of 70 to 100 LPA.
- Opportunities for international travel may be available.