Deltek
Website:
deltek.com
Job details:
Position Responsibilities:
Deltek is seeking a skilled specialized Automation Engineer with expertise in testing and test automation of AI-powered workflows and intelligent features across ERP product lines.
Responsibilities
- Develop end-to-end test strategy for AI features across ERP modules such as Finance, Supply Chain, Projects, HR, and Payroll
- Execute functional, regression, scenario-based, and exploratory testing for AI-enhanced workflows
- Validate AI/ML outputs including:
- Prediction accuracy & scoring
- Recommendation relevance
- Natural-language responses
- Behavior consistency across edge cases
- Create and maintain automation suites for UI/API and AI validations
- Design AI safety test cases → hallucination detection, bias testing, ethical and compliance alignment
- Collaborate with Product Managers, AI/ML engineers, SMEs, and Developers to define release quality gates
- Ensure test coverage of data pipelines: input validation, data sampling, transformations, and output visualizations
- Contribute to performance, scalability & data-drift testing of AI features in ERP contexts
- Maintain test documentation — test plans, test cases, defect logs, and sprint reports
- Drive automation adoption in CI/CD pipelines for AI-enabled releases
- Provide mentorship to senior engineers and teams, guiding best practices and fostering professional development.
Qualifications & Experience:
- 6+ years of proven experience in automation testing, with at least 2+ years working with AI or ML-based systems
- Proven track record of designing and implementing AI Based automation solutions.
- Expertise in multiple automation tools and frameworks, with a deep understanding of the latest technologies.
- Experience in developing and refining overall strategies for API testing.
- Proficiency in using version control systems like Git for managing code.
- Experience with cloud-based ERP solutions.
- Experience with performance and security testing.
- Strong debugging and analytical skills.
Good-to-Have Skills
- Understanding of LLM fine-tuning, vector databases, RAG pipelines, embeddings
- Familiarity with data privacy & compliance frameworks
- Experience with security testing for AI systems
- Exposure to AIOps or self-healing automation frameworks.
Technical Qualifications:
- In-depth understanding of modern QA methodologies, tools, and processes with a focus on AI-driven features and intelligent workflows.
- Strong understanding of API architectures including REST, SOAP, GraphQL, and WebSocket, along with experience validating AI-generated responses via APIs (e.g., LLM response endpoints, inference APIs).
- Hands-on experience with API testing tools such as Postman, SoapUI, JMeter, or REST Assured, including automated evaluation of AI output scoring / JSON inference payloads.
- Proficiency in one or more programming languages — Java, Python, or JavaScript — with the ability to build AI output validation utilities, automated prompt-testing harnesses, or data-driven frameworks.
- Knowledge of CI/CD pipelines (Jenkins, Azure DevOps, GitHub Actions) and automation integration for AI feature regression and model-response validation.
- Familiarity with LLM / ML concepts from a QA perspective — prompt testing, hallucination detection, prediction scoring, explainability validation, bias checks, guardrail/output sanitization.
- Strong SQL and database testing skills, including validation of training datasets, feature store data, and end-to-end AI data pipelines.
- Good understanding of Agile / Scrum development practices with experience participating in AI model release cycles (shadow mode, AB comparison, fine-tuning validation, or phased rollout).
Soft Skills:
- Excellent communication and collaboration skills.
- Ability to manage multiple tasks/projects concurrently.
- Strong interpersonal skills for working in a team-oriented environment.
- Excellent problem-solving skills and attention to detail.
- Ability to work independently and as part of a team in a fast-paced environment.
- Proactive and self-motivated with a strong desire for continuous learning and improvement.
Click on Apply to know more.