Williams-Sonoma
Website:
williams-sonomainc.com
Job details:
Job Title: Performance Test Architect (Strategy, Design & AI-Driven Engineering)
Role Summary
We are seeking a highly experienced Performance Test Architect to lead the performance engineering strategy during the design phase of application development. This role is responsible for gathering performance requirements from multiple stakeholders, translating them into a comprehensive testing strategy, and defining scope, timelines, environments, and reporting frameworks.
The role requires strong expertise in leveraging AI and agentic automation capabilities to enhance performance testing, including intelligent workload modeling, automated script generation, predictive analysis, and AI-driven reporting. The ideal candidate combines deep performance engineering knowledge with practical AI implementation experience to drive efficiency and innovation.
Key Responsibilities
1. Performance Strategy, AI Integration & Design Leadership
- Lead performance engineering initiatives starting from requirements and design phase.
- Gather performance requirements from:
- Business stakeholders (SLAs, SLOs, KPIs)
- Product Owners
- Architects
- Infrastructure and SRE teams
- Leverage AI tools (e.g., OpenAI API, Claude) to:
- Interpret requirements and extract performance scenarios
- Assist in defining workload models and test strategies
- Define:
- Performance testing strategy
- Test approach (load, stress, soak, spike, scalability, failover)
- Entry and exit criteria
- Performance acceptance benchmarks
- NFR traceability matrix
- Identify performance risks early using both traditional analysis and AI-assisted insights.
2. Workload Modeling, Capacity Planning & Predictive Analysis
- Translate business workflows into workload models using:
- Production data
- AI-assisted pattern recognition and forecasting
- Define:
- User concurrency
- Transaction mix
- Peak vs. average load
- Growth projections
- Build predictive models to identify potential bottlenecks (e.g., memory leaks, thread contention) early in the cycle.
- Provide capacity planning inputs and infrastructure sizing recommendations.
3. Test Planning, AI-Driven Execution & Offshore Coordination
- Create detailed performance test plans including:
- Scope
- Test durations and cycles
- Environment requirements
- Test data strategy
- Use AI to:
- Generate and maintain performance test scripts from API specs (Swagger/OpenAPI)
- Auto-heal scripts when API contracts change
- Coordinate with offshore Performance Engineers to:
- Execute performance tests
- Monitor runs and validate outputs
- Act as quality gate for all deliverables and ensure AI-generated artifacts are validated before use.
4. Tooling, Observability & Autonomous RCA
- Define and standardize performance testing and observability architecture.
- Integrate AI-driven workflows with APM and logging tools to enable:
- Autonomous Root Cause Analysis (RCA) through correlation of telemetry and logs
- Build agentic solutions using frameworks such as LangChain or CrewAI to:
- Analyze performance data across system layers
- Generate bottleneck hypotheses
- Establish observability across:
- Application
- Middleware
- Database
- Infrastructure
5. Reporting, Dashboards & AI-Driven Insights
- Define performance reporting frameworks:
- Real-time dashboards
- Test reports
- Executive summaries
- Leverage AI to:
- Auto-generate reports and summaries
- Highlight anomalies and trends
- Provide actionable insights and tuning recommendations
- Enable natural language querying of performance results where applicable.
6. Test Data Strategy & Synthetic Data Generation
- Define and manage performance test data requirements.
- Leverage Generative AI to:
- Create high-quality synthetic datasets
- Mimic production-like distributions
- Ensure compliance with data privacy regulations (e.g., GDPR)
- Ensure test data supports realistic and scalable test scenarios.
7. Self-Service Performance Engineering Enablement
- Drive adoption of a self-service performance testing model by:
- Integrating AI-enabled workflows into CI/CD pipelines
- Enabling developers to trigger automated performance tests
- Supporting autonomous performance validation for early-stage builds
8. Governance, Standards & Continuous Improvement
- Establish performance engineering standards and reusable frameworks.
- Define best practices for:
- AI-assisted performance testing
- Validation and governance of AI-generated outputs
- Drive shift-left performance practices and continuous optimization.
9. Offshore Team Collaboration
- Break down strategy into executable tasks for offshore teams.
- Provide guidance on both:
- Performance testing practices
- Usage of AI-driven tools and outputs
- Conduct reviews and ensure adherence to quality and timelines.
- Act as escalation point for complex issues.
Required Qualifications
Experience
- 10+ years in performance testing/performance engineering.
- 3+ years in architect-level or strategic role.
- Strong experience in design-phase performance strategy definition.
- Experience working with offshore delivery teams.
Technical Skills (Core + AI Integration)
- Strong expertise in performance testing tools and methodologies.
- Hands-on experience integrating AI into engineering workflows using:
- OpenAI API
- Claude
- Experience with agentic frameworks such as:
- LangChain
- CrewAI
- Proficiency in Python for automation and orchestration.
- Strong understanding of:
- APM/observability tools
- Distributed systems and microservices
- APIs and messaging systems
- Cloud platforms (AWS/Azure/GCP)
- Experience integrating performance testing into CI/CD pipelines.
- Ability to validate and govern AI-generated outputs.
Soft Skills
- Strong stakeholder management and requirement gathering skills.
- Excellent documentation and presentation abilities.
- Ability to translate business SLAs into technical metrics.
- Strong analytical and problem-solving skills.
Preferred Toolset
- Performance Testing: JMeter
- Observability/APM: AppDynamics
- Messaging/Streaming: Kafka
- CI/CD: Jenkins
Preferred Qualifications
- Experience in retail, e-commerce, or high-volume transaction systems.
- Exposure to SRE practices (SLIs/SLOs/SLAs).
- Experience with large-scale distributed architectures.
- Experience implementing automation frameworks or internal tools.
Success Criteria in This Role
- Clear, well-defined performance strategies aligned with business goals.
- Early identification and mitigation of performance risks.
- High-quality workload models and test plans.
- Effective use of AI to reduce manual effort and improve speed.
- Efficient offshore execution with minimal rework.
- Actionable reporting with measurable impact on system scalability and reliability.
Click on Apply to know more.