Will shift-right testing leave shift-left behind?

I heard from a development leader that some big-time analyst was recommending his firm turn its sights on ‘shift-right testing.’ 

I must admit, the first time I heard ‘shift-right testing,’ it sounded abominable to me — and not just because it came from someone else’s ivory tower. Shift-right did exist before. 

Digging around a bit, I can find a few examples of shift-right testing mentioned by others in Agile/DevOps circles in the last 5 years or so, but for the most part it didn’t seem to be a thing anyone wanted to do until mid-last year.

I’ve spent most of my professional life convincing businesses to shift things left — shift-left testing for software, shift-left demand and supply forecasts for supply chains, shift-left analytics to understand future implications earlier than your competition.

Hopefully that explains why it seems heretical for me to talk about shift-right testing at all. 

Will shift-right testing somehow cheapen shift-left testing, making it old news? Or could it cause some teams to stop early preventative testing, just like viral shared videos can prevent some otherwise rational people from getting vaccinations?

Give up, it’s going to happen anyway

With CI/CD automation, DevOps practices and cloud-native delivery of software, the software pipeline is moving at such breakneck speeds now that shift-right is a near inevitability.

Given how software development incentives are so often aligned with delivering features faster rather than ensuring complete and early testing, I don’t expect many organizations will let shift-left testing failures hold up release cycles for long.

So what do we do now, allow our customers to become software testers?

It seems, no matter how much we try testing earlier in the software lifecycle, with greater automation, there will always be too much change and complexity to prevent all defects from escaping into production — especially when the ever-changing software is likely executing on ephemeral cloud microservices and depending on calls to disparate APIs.

There are several interesting vendors that offer pieces of the shift-right puzzle, and to their credit, none really touch the third rail of saying ‘you don’t need QA teams,’ or calling themselves ‘shift-right testing.’ That’s smart marketing.

And it doesn’t really matter, they can call it progressive delivery. Canary releases, blue/green deployments, feature flagging, real user monitoring, and even some observability, chaos engineering and fast issue resolution workflows. All things that advanced teams can automate to improve quality and performance nearer to production, and even post-delivery.

The thing that’s making shift-right practices stick now, besides engineers working from home for a few months? With cloud compute infrastructure and storage costs going down, companies can now afford to drive more ephemeral workloads out than ever, and monitor and automate post-deployment reactions to exponentially larger data sets.

Shift-Left, but Bank-Right

Much like a bike on a velodrome, or a NASCAR race track banking around the left turns — shift-right testing has less to do with validating what the software does, and more to do with accounting for everything the software might do under stress.

Not to be dogmatic, but I don’t consider it testing to put trial releases in front of perhaps smaller groups of customers who aren’t being told they are beta testing. I wouldn’t want to be holding a pager when a graduated release doesn’t blow up until it scales to half my user base, either…

It is, however, quite valid to call it validation. Or — maybe risk mitigation. Damage control. Blast radius reduction. Those are all great shift-right aspects of operational excellence to strive for.

When you are shifting right you aren’t really shifting testing at all, you are banking the track. 

You are engineering more tolerance into the system. 

Bank-Right and build more operational tolerance into your release track, so you can afford to Shift-Left more testing and release automation, and go even faster.

You still need early testing, but at this velocity, all the testing in the world will never reach the asymptote of 100% perfection in production. Bank-Right approaches offer slopes and guardrails to keep the race on the track, and put out fires faster, even if the racers behave abnormally. 

You might even put shift-right security testing into this high-tolerance bucket, both for shutting down known threats that reappear due to misconfiguration or stolen credentials, as well as reducing the harm of unknown or Day0 exploits that teams couldn’t have possibly shift-left tested for.

The Intellyx Take

Shift-Left and Bank-Right go hand in hand, just like design and engineering in the real world. 

When you drive on a bridge, you hope that it was designed and tested using simulations and scale models to flex gracefully when confronted with a variety of natural forces and traffic contingencies. 

You would also want that bridge to be engineered to provide early warning signs and continuously monitored post-production, with failsafes to mitigate risk and reduce harm if anything does go wrong.

Ultimately, we’ll see both approaches as two different lenses for improving customer experience, no matter what they are called.

I think I’ll do some more laps on this topic for my upcoming writing.

• Hey you! Do you have any interesting observations for me on Shift-Left? Hit me up at pr@intellyx.com and I may feature you in my next article.

 

© 2021 Intellyx LLC. Intellyx publishes the Cloud-Native Computing Poster, the weekly Cortex and Brain Candy newsletters, and advises business leaders and technology vendors on their digital transformation strategies. Intellyx retains editorial control over the content of this document. No parties mentioned in this story are Intellyx customers. Image composite credits: Sayam Gearspec, Erik F. Brandsborg, flickr CC2.0 license.