About the role
Role Overview
We are looking for a Data Engineer with 3–4 years of experience to design, scale, and optimize data infrastructure, experimentation platforms, and AI tooling. In this role, you will bridge the gap between traditional data engineering, scalable MLOps, and rapid GenAI enablement, directly accelerating how cross-functional teams build, evaluate, and deploy data products.
About:
Fletch is a U.S. based technology firm building an open insurance ecosystem connecting consumer apps to insurance providers. The team has previously built global payment infrastructure at scale (processing $billions in transactions) and founded enterprise
technology companies with solutions in commercial use by over 100 corporates. Fletch has seen rapid traction (existing partnerships with leading U.S. insurance providers) and is backed by top global fintech VCs and angels. The team is now looking to scale further hiring & expansion.
Location: Remote
Key Responsibilities
* Data Science & ML Platform Engineering:
* Design, scale, and maintain a centralized Data Science Platform to standardize machine learning workflows across the organization.
* Build reusable CI/CD pipelines, containerized environments, and feature stores to accelerate end-to-end model experimentation, testing, and deployment.
* Establish governance, observability, and reproducible pipeline templates for data science and engineering teams.
* Experimentation & A/B Testing Infrastructure:
* Build and maintain robust data pipelines supporting company-wide A/B testing, multi-armed bandits, and multivariate experimentation frameworks.
* Implement automated sample-sizing, randomization, hypothesis validation, and metric aggregation pipelines (e.g., CUPED, guardrail metrics, p-value calculations).
* Partner with product, analytics, and engineering teams to ensure rigorous experiment logging, and reliable metric computation.
* AI Model Evaluation & Performance Optimization:
* Develop and implement automated evaluation algorithms and benchmarking suites for AI/ML models to improve operational efficiency and runtime performance.
* Implement automated regression testing, drift detection, and offline/online evaluation loops (monitoring accuracy, latency, compute cost, and throughput).
* Standardize model scorecard frameworks to evaluate production readiness and prevent performance degradation.
* GenAI Tooling & Rapid Prototyping:
* Rapidly prototype and deploy GenAI-powered internal tools and services for business stakeholders, including marketing, customer support, and data analytics teams.
* Automate insight extraction from structured and unstructured data sources using LLMs, embeddings, and vector databases.
* Build features and interfaces that enhance model explainability (e.g., SHAP/LIME integration, automated narrative summaries), making complex AI decisions actionable for non-technical users.
* Data Modeling & Core Engineering:
* Architect resilient batch and real-time streaming data pipelines using SQL, Python, and modern data stack orchestrators.
* Optimize data lakehouse/warehouse query performance, partitioning strategies, and compute resource allocation.
Required Skills & Qualifications
Technical & Platform Engineering
* Languages: Advanced proficiency in Python, DynamoDB and SQL, with an emphasis on writing modular, memory-efficient, and maintainable code without relying on heavy distributed infrastructure.
* Data Processing & Orchestration:
* Hands-on experience with lightweight, high-performance in-memory processing engines such as DuckDB and Polars.
* Production experience with modern open-source workflow orchestrators like Dagster or Prefect for managing data pipelines.
* Practical usage of dbt Core or SQLMesh for transformation modeling, testing, and documentation.
* MLOps & Platform Architecture:
* Experience setting up and maintaining self-hosted or lightweight MLOps tools, specifically MLflow for experiment tracking and model registries.
* Containerization and deployment using Docker and Docker Compose, with API serving via FastAPI or BentoML.
* Familiarity deploying services on compute (e.g., AWS ECS/Fargate).
Experimentation & Analytics
* A/B Testing Methodologies:
* Strong foundation in statistical hypothesis testing (A/B-tests).
* Experience implementing variance reduction methods (such as CUPED) using Python libraries (SciPy, Statsmodels, NumPy).
* Familiarity with experimentation and feature-flagging platforms such as GrowthBook or PostHog.
* Analytics Engineering:
* Ability to transform raw application and telemetry logs into clean dimensional models using dbt Core.
* Experience configuring and building stakeholder dashboards on open-source BI platforms such as Metabase, Apache Superset, or code-based BI like Evidence.
AI Evaluation & Generative AI
* Evaluation Algorithms:
* Experience setting up programmatic evaluation pipelines using open-source frameworks like Ragas, DeepEval, or custom LLM-as-a-judge scoring functions.
* Implementation of automated regression testing for accuracy, response latency, token usage, and hallucination detection.
* GenAI Development & Fast Prototyping:
* Experience developing end-to-end prototypes using orchestration frameworks (LangChain, LlamaIndex) and cost-routing proxies (e.g., LiteLLM).
* Ability to rapidly deliver stakeholder-facing internal tools and UIs using lightweight application frameworks like Streamlit or Gradio.
* Model Explainability & Insights:
* Experience applying explainable AI libraries (e.g., SHAP, LIME) to interpret model behaviors for business stakeholders.
* Practical ability to parse unstructured data (documents, support tickets, web pages) using open-source parsing libraries (e.g., pdfplumber, Beautiful Soup).
This page is fully interactive when JavaScript is enabled. Please enable JavaScript to apply or browse related roles.
