Principal GenAI Systems Engineer

This is a remote position.

We are seeking a Principal GenAI Systems Engineer to join our team. Y ou will be responsible for designing, developing, and deploying applications that leverage generative AI models. You will work closely with company founders, machine learning engineers (DataML Engineers) and software engineers to develop a specialized and scalable generative AI application interfacing with telemetry-focused AI products. Your role will involve both front-end and back-end development, ensuring seamless functionality and performance consistency for a unique implementation of Retrieval Augmented Generation (RAG). A significant aspect of this role will involve architecting, engineering, implementing and testing system prompting configurations and pipelines, which are essential for unlocking the vast insights of company AI products for downstream automated Actions and semi-autonomous Agents.

Responsibilities:

• Design, configure and optimize the GenAI-tech stack including: LLM, Vector DB, Encoder / Decoder, Vector Search (ANN, HNSW), prompt framework and supporting cloud compute and service resources.
  
• Design and implement RAG pipelines that enhance generative AI models by integrating external data sources.
  
• Architect and engineer efficient retrieval systems that can fetch relevant data from databases, knowledge graphs, or external APIs to augment AI-generated responses.
  
• Develop prompting pipelines that leverage context and retrieved information to generate accurate and contextually relevant responses.
  
• Collaborate with machine learning engineers to implement advanced techniques such as vector search, semantic search, and embeddings to improve data retrieval accuracy.
  
• Build and maintain robust pipelines for data retrieval, preprocessing, and integration into the generation process.
  
• Implement automated testing frameworks to validate the performance of RAG and prompting pipelines.
  
• Ensure that the retrieval and generation pipelines are scalable, reliable, and maintainable.
  
• Continuously monitor and refine pipelines to improve efficiency and reduce latency.
  
• Implement monitoring, logging, and alerting to maintain system health and uptime
  
• Collaborate with cross-functional teams including UX/UI designers, product managers, and DevOps engineers to deliver high-quality products.
  
• Collaborate with DataML Engineers, Integration Engineers & GenAI Engineers for customer-specific deployments & configurations
  
• Write clean, maintainable code and conduct code reviews.
  
• Document technical architecture, processes, and best practices.
  

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  
• A strong foundation in software engineering principles is essential.
  
• Additional coursework or certifications in AI/ML or data science is a plus.
  
• 5+ years of professional experience in complex systems engineering, with a strong focus on AI-driven applications.
  
• Proven experience in integrating and deploying machine learning models, particularly in generative AI (e.g., GPT, GANs, VAE, etc.).
  
• Demonstrated experience in architecting, engineering and deploying RAG pipelines for generative models and complex prompting systems.
  
• Familiarity with Python-based APIs.
  

Advanced Qualifications - Nice to have:

• Masters degree in Computer Science, Software Engineering or a related field.
  
• Experience with scalable and high-performance application development in a cloud environment (AWS, GCP, Azure).
  
• Familiarity with technologies and/or data architectures such as: Product Analytics (e.g Pendo, Mixpanel), and Observability systems (e.. Grafana, New Relic, Dynatrace).
  

Benefits

• Work Location: Remote
  
• 5 days working