I build AI-powered systems
that eliminate toil,
scale quality, and ship faster.
10+ years building test infrastructure and AI systems for enterprise products. Production RAG pipelines, semantic routing, GPU inference, and automation frameworks that serve 100+ engineers — and actually work in production.
About
I'm a Staff-level engineer at the intersection of AI systems and quality infrastructure. Over 10+ years I've moved from building test frameworks to designing RAG pipelines, semantic routing layers, and GPU inference infrastructure — while still owning the quality strategy and automation architecture that lets teams ship without fear.
I think in systems. I care about what breaks in production, not just what passes in CI. And I build things that other engineers can actually use and maintain. Currently: open to Staff / Principal roles in AI-forward engineering orgs. Remote · Montenegro · EU-friendly timezones.
What I Do
- AI Systems & RAG Infrastructure — Design and build production RAG pipelines: hybrid dense+sparse retrieval, RRF reranking, pgvector HNSW indexing, semantic multi-agent routing. From architecture to deployed inference — end to end.
- GPU Inference Infrastructure — Self-hosted model inference on AMD ROCm, bfloat16 weights, batch encoding pipelines, VRAM budget management. 10-30x throughput vs sequential baseline.
- Test Automation Architecture — Playwright, WebdriverIO, Appium, Flutter integration tests — frameworks that serve 20+ teams and 100+ engineers without falling apart. From greenfield to enterprise scale.
- Quality Strategy & CI/CD — Quality gates that catch issues before production. Not "more tests" — the right tests, in the right places, integrated into the pipeline that teams actually use.
- Team Leadership & Mentoring — Built and grew QA automation teams from 0. Architecture reviews, code standards, hiring, onboarding. 5-10 engineers mentored across multiple concurrent teams.
Technical Depth
Not a checklist of logos. These are the tools I use daily to ship production AI systems and test infrastructure at scale.
AI / ML Systems
Production RAG, inference, semantic routing
Test Infrastructure
Enterprise frameworks serving 100+ engineers
Infrastructure
What keeps the systems running in production
Languages & Frameworks
The foundation everything else builds on
Depth over breadth — every tool listed here is one I've used in production, not just tutorials.
Core Stack
Python · PyTorch · ROCm · FastAPI · PostgreSQL + pgvector · Redis · asyncio · tree-sitter · Anthropic Claude API · OpenAI API · Genetic Algorithms · HNSW · BM25 · RRF · RAGAS
Also worked with: TypeScript · Playwright · WebdriverIO · Appium · Detox · Flutter Integration Tests · Docker · GitLab CI · AWS · Pact · Kubernetes
From Junior to Lead
10+ years of continuous growth in QA Automation Engineering
IT Light
Junior QA Automation Engineer
Started automation journey building first test suites
IT Light
Junior QA Automation Engineer
Started automation journey building first test suites
Tamga Digital
QA Automation Engineer
Built automation frameworks for enterprise clients
Tamga Digital
QA Automation Engineer
Built automation frameworks for enterprise clients
Revenue Grid
Middle QA Automation Engineer
Led test architecture for SaaS platform
Revenue Grid
Middle QA Automation Engineer
Led test architecture for SaaS platform
EPAM Systems
QA Automation Engineer
Enterprise-scale automation & team leadership
EPAM Systems
QA Automation Engineer
Enterprise-scale automation & team leadership
Favbet Tech
QA Automation Lead
Leading QA automation strategy & team
Favbet Tech
QA Automation Lead
Leading QA automation strategy & team
Experience
10+ years of hands-on engineering and leadership experience in test automation, CI/CD integration, and quality strategy for web & mobile products.
Remote / Hybrid
Tech Stack Deep Dive
Key Responsibilities
- Built and led a TypeScript/Playwright-based automation framework for web & mobile, increasing regression coverage to 80%+ and reducing release time
- Integrated E2E and API tests into CI/CD (GitLab / Bitrise / AWS), catching critical issues before production and cutting production incidents
- Defined quality strategy and critical user journeys together with engineering and product, aligning testing efforts with business impact
- Mentored and coached 5-10 QA engineers, introduced code reviews, coding standards and shared testing practices across teams
- Led mobile automation implementation for iOS and Android using Appium and Detox, stabilizing release cycles
Achievements
- 80%+ Regression coverage achieved for web & mobile products
- Quality gates Integrated into CI/CD pipelines preventing risky releases
- 5-10 engineers Mentored and coached across multiple QA teams
- Mobile automation Introduced for iOS and Android, reducing manual testing cycles
- Internal RAG system Confluence/Jira/Figma indexed, hybrid search deployed, ready for org-wide rollout
Remote — Built production RAG system from scratch for a B2B SaaS platform. Designed end-to-end: from data ingestion to query serving.
Tech Stack Deep Dive
Key Responsibilities
- Designed hybrid retrieval pipeline: dense vectors (cosine similarity) + BM25 sparse retrieval, fused with Reciprocal Rank Fusion reranking
- Built semantic routing layer matching queries to 30 specialized expert agents via 4-axis cosine similarity encoding
- Developed embedding optimization harness with genetic algorithms: population-based search, single-point crossover, keyword-level mutation
- Self-hosted GPU inference: AMD RX 7900 XT (20GB VRAM), ROCm, bfloat16 weights, batch encoding pipeline
- Designed HNSW vector index schema in pgvector for sub-10ms retrieval at scale
Achievements
- 100% hit_rate on 101 validation queries across 30 agent categories
- +15% Embedding similarity via iterative genetic optimization (14 experiments)
- 10-30x Inference speedup via GPU batch encoding vs sequential baseline
- Production hybrid search BM25 + HNSW + RRF reranking fully deployed
Expertise & Technology Stack
A curated collection of tools and frameworks mastered over 10+ years of building enterprise-grade automation solutions.
Test Infrastructure
Enterprise-scale frameworks serving 100+ engineers.
Programming Languages
Core engineering capabilities.
CI/CD & DevOps
Infrastructure & Pipeline integration.
Testing Strategy
Comprehensive QA approaches.
Tools & Platforms
Essential ecosystem mastery.
Leadership
Team building & Strategy.
AI & ML Engineering
Building production AI systems: RAG pipelines, semantic routing, embedding optimization, and GPU inference infrastructure. Combining ML research with engineering rigor.
RAG Pipeline Architecture
Hybrid search with dense vectors (cosine similarity) + BM25 sparse retrieval, fused with Reciprocal Rank Fusion reranking
Embedding Optimization
Iterative benchmark-driven optimization with genetic algorithms — response-space prompt phrasing for retrieval alignment
Multi-Agent Routing
Semantic routing layer matching queries to specialized expert agents via 4-axis dense vector encoding
GPU Inference Infrastructure
Self-hosted inference on AMD ROCm with bfloat16 weights, VRAM budget management, and batch encoding pipelines
AI Tools & Platforms
AI Integration Achievements
- Built production RAG pipeline with hybrid vector + keyword search and RRF reranking
- Achieved 100% hit_rate on 101 validation queries across 30 agent categories via semantic routing
- Improved embedding retrieval alignment by 15% through iterative optimization with genetic algorithms
- Self-hosted GPU inference with 10-30x batch encoding speedup on AMD RX 7900 XT (20GB VRAM)
Embracing AI to Build the Future of QA
I actively integrate AI tools into QA workflows, from test generation to code review, helping teams work smarter and deliver faster.
GitHub Activity
478 contributions in the last year
478 contributions in the last year
Top Languages
Selected Projects
Key automation and quality initiatives showing architect-level thinking and hands-on delivery.
Studies
Rate
Focus
Enterprise E2E Automation Framework
Built enterprise test automation platform from scratch serving 20+ teams and 100+ engineers. Reduced regression cycle 90% (3 weeks → 3 days) saving $180k/year. Eliminated 80% of integration tests through contract testing while achieving 95% coverage.
The Challenge
The organization faced a critical challenge: multiple applications built with different technology stacks (React, Angular, Vue) had no unified testing approach. Each team maintained their own testing solutions, leading to inconsistent quality gates, duplicated efforts, and a 3-week regression cycle that was blocking rapid delivery.
The Solution
Architected a modular framework with TypeScript + Playwright as the core, supporting multiple test runners
Impact Analysis
Quantifiable results achieved post-implementation
More Case Studies
What Colleagues Say
Feedback from team members and colleagues I've had the pleasure to work with at Favbet Tech.
For Hiring Managers
I'm looking for Staff / Principal Engineer roles where:
- AI infrastructure is a first-class concern, not an afterthought
- Architecture decisions matter more than ticket velocity
- The team wants to build systems that scale — not just pass sprints
- Quality is owned by engineering, not delegated to a QA department
What I bring:
- Production RAG systems and AI infrastructure — not toy projects
- Test automation that served 100+ engineers across 20+ teams
- 10 years of knowing exactly where quality breaks in real products
- The ability to go from architecture whiteboard to deployed system
Remote · Montenegro · EU-friendly timezones · Open to async-first teams
Contact
Open to Staff / Principal roles in AI-forward engineering orgs. Let's connect:
LinkedIn: linkedin.com/in/andrew-peretyatko
Location: Montenegro (open to remote / EU-friendly time zones)