Push for QA as your devex powerhouse

Your dev team is coding faster than ever, introducing exponentially more entropy, it’s time to modernize QA

Aug 20, 2025

tl;dr: As AI progresses, the bottleneck shifts to testing fast & well. We argue for splitting the traditional QA role into two strategic functions: QA Analysts, who ensure you build the right thing for the customer, and SETIs, who build infrastructure for devs to ship safely and fast.

Your team is coding faster than ever, and now what was a team of 10 feels like a team of 50. The CEO wants to go even faster, and that has resulted in your CTO pushing for bigger releases, still every few days, maybe a couple of weeks. Flash-forward 2 months, and now you are sitting with Customer Support & the Account Management team. The quality of your flagship product is dropping, and a top customer is about to churn. Now you are in a battle of wits with Product, Customer Support, Sales and Engineering. The blaming dance commences.

Have you seen the blaming dance? I’ve been there. QA says they didn’t have time to review, Engineering says Product wanted it to be released ASAP, Product points at Sales, at best, or back at QA at worst. Sales says ‘isn’t it your job to deliver things right?’. You walk out of the meeting with the certainty that your team is now going to be only doing Quality Control. There’s no time to code, so you have gone back to a team of testers.

You wish it would end on that Defect Escape Rate1 increase since you started using Cursor, it’s not. It seems your leaders think with AI you can do more with less, it’s time to reduce your department costs. So not only are you seeing your DORA metrics spike, you need to do more with less. This isn’t a feeling, it’s a trend, e.g. the 2024 DORA report found that while AI boosts individual productivity, it has been linked to a 1.5% reduction in delivery throughput and a 7.2% decrease in delivery stability.

Personally, I’ve been faced with challenges like this in QA, HR, Data, SRE, Growth and many other orgs, and I’m going to tell you what worked in all cases: reframe the problem, think big, think positive. All that summarizes in a single question:

How can you make a power-center out of QA?

Each of those times, I found these 3 questions helped me distill what we needed next:

(1) What are you doing that you shouldn’t be doing?
(2) What should you actually be doing to have more impact long term?
(3) How can you shift motivations in your team to make (1) & (2) happen.

Below you can see this plan laid out to move QA beyond QC, a plan that took me 1yr to implement with a 60 people team before the AI revolution; be patient.

First: Fight the Illusion of Quality Control.

If your organization still thinks of QAs as testers, or simply release gatekeepers, it’s key to explicitly name those activities. You need to equate Quality Control activities with ‘toil’ for the engineering team. The key here is to show your peers that these activities are a particular implementation of a broader vision of Quality Assurance. Avoid stepping into exactly what QA is, for now focus on naming each Quality Control activity your team does.

Start by building a list of any and all repetitive tasks that happen once the code has been written, e.g. writing tests, lint, image diff, static code analysis, build, deploy in staging, canary, feature flags set up, etc. Armed with that list, bring your SRE or DevOps team onboard so together you can have a shared, unified, understanding of what are Quality Control activities in your company. For example, you could say Quality Control groups staging monitoring, alerting, performance, all testing, static code analysis, etc. Believe me, they’ll be excited to see you understand the pain of their daily work, and how their time should really be invested in building better infrastructure.

With SREs as allies, start small, educating teams, presenting the difference between Quality Control and Quality Assurance. Add some Engineer Managers to the conversation, and introduce a key core concept: Accountability. At this point you need to have a discussion on who will be accountable for Quality Control. This is why the SRE team can help you, they implement the tools for logging, monitoring, alerting, etc, but Engineering uses those frameworks. Quality Control should center on the same, you provide the tools you don’t enforce the implementation.

At this point, naturally, people are going to start asking themselves what is Quality Assurance; which leads us to step 2.

[Note: It’s important to know you are defining Quality Control for your Company, and you could use arbitrary terms, like Accidental QA, if that simplifies conversations].

Second: Frame QA as an Engineering org-wide Practice

“Quality Assurance is meant to prevent mistakes and defects in the development and production by avoiding problems and delays when delivering products or services to customers”.

You need to make an effort to remind everyone about a key aspect of Quality Assurance, it comprises avoiding delays not just problems, and customer satisfaction. The defect prevention aspect differs from the defect detection aspect of Quality Control and it is what has been referred to as a shift-left.

You should harness the core concept of shift-left, not into human hours saved by detecting a defect early, but by the reputation damage and potential loss-of-revenue that could result from building and shipping the wrong product. I recommend you use 2 data-points to accentuate the message:
(a) In 2024 alone, these costs were estimated at 2.4Tn in the US alone, the pain is real. You can find similar numbers for whichever geo you are focused on.
(b) Approach customer success & sales, ask them how often they need to waive fees, give discounts, or lose customers because something wasn’t shipped in the right way; either late, faulty, or improper solution.

Those two above translate immediately to mission critical business needs, defined as a need for effective and efficient product delivery.

The final step in your presentation is to translate QA into two key areas that bring value to the company:

Quality Analysis (Essential QA)
Working closely with Product and go-to-market to understand the customer needs.
QA needs to understand beyond features, the quality and the service expected from customers. That way you can define guardrails for the organization, incl. non-functional requirements, that will ensure your releases are effective.
Quality Control (Accidental QA)
Working closely with devex to accelerate delivery in the ‘right way’. QA is also responsible for providing the tools and frameworks for efficient releases. You’ll collaborate closely with SREs, Engineering and UX to provide that infrastructure that would guarantee fast releases with the right quality, or even faster rollbacks.

We have defined a new scope, it’s time to re-shape QA to have impact across the organization.

Third: Implement The QA Executive Playbook

You’ve done it! Everyone understands how Quality Assurance has a broader scope and it shouldn’t be a gatekeeper, instead it should be an enabler. Now, let’s outplay Tech!

You need to bring it all together with flawless execution, to have company-wide impact and become a powerhouse. For that, you need to ensure you have top performers building strong foundations in each area. Timing is of the essence, every effort you do during the next 3 months needs to be successful so your credibility grows, and it’s going to be impossible to do this alone.

Building bridges
A Quality Assurance Analyst needs to work extremely closely with Customer Support, UXR, Account Managers, Product Managers, and any other person that works directly with customers. If you can spend 10hrs a week listening into customer calls, do it! Your team needs to become a subject matter expert on the needs and expectations from customers. (My hack: Listening to Gong Calls instead of podcasts)

Please make sure to have a clear scope early on: focus on understanding what are those customer needs above all, much less on how to solve for those. This team needs to fall in love with the problem, as solutions should ‘surprise them’ .

Two KPIs to track here: NPS (esp. for new features), and # valid tickets from customer support (think of DER).
Delegating until it hurts
Your Software Engineer in Test & Infrastructure team (SETIs) should invest all their time on building the infrastructure so software engineers can implement their own tests, and their managers have visibility on how well they are doing so.

You can always start with quick wins like adding Image Diff, improving record-replay API tests, or adding a new low-touch, no-code, provider.

The KPIs I like to start with are proxies for change-lead-time, and change-failure-rate.

The-one-thing: No-one should be a tester.

Stop asking for your testers to do better and faster, instead teach them how to have exponential impact. Divide them in at least two groups, those that have affinity with technology, and those that have affinity with the product. The former should be trained to become SETIs, working closely with SRE/DevOps, and driving delivery acceleration. The latter need to transition to QA Analysts, with a deep comprehensive understanding of the customers and company vision. They should center on user satisfaction.

Doing only one will render you exposed to the whims of your leaders, and if you manage to do both, congrats! You are on an executive path!

References

DER – Defect Escape Rate measures the number of undiscovered defects that escaped detection in the product development cycle and were released to customers. DER = (Defect Escapes /Total Defects)∗100