had checkout humming for years, one processor, never thought about it. then my processor had a regional outage for a few hours and my checkout just threw errors the whole time, every attempt in that window lost, and i had no fallback and no graceful message, just a broken page. a longer outage wouldve been brutal. so genuine question not a lecture, how are you handling a payment processor going down at checkout, do you actually have a fallback or a graceful degrade, or is it one processor and hope like i had
oof the silent single point of failure, ive been there. what i do now is a second processor configured and ready to flip to, even if nearly all traffic normally goes to the primary. you dont want to be integrating a backup gateway live during an outage, thats the worst possible time to be reading api docs. have it wired and tested before you ever need it.
This is one of the most skipped things and the most expensive when it bites. I automated the failover so a health check flips checkout to the backup processor without me awake at the keyboard, otherwise it never happens in time because youre asleep or panicking. if switching processors is a manual scramble you wont do it fast enough, so make the switch automatic and then test that it actually triggers.
the other half nobody plans for is the failed payment itself. even with a fallback, some charges will error mid outage, so queue the attempt and a clear retry message instead of a dead error page, otherwise you lose the customer and the sale both. now i know a declined transaction during an outage gets retried and the user gets told, not just dropped on the floor.
yes i hadnt even thought past just having a second processor, i was so focused on the gateway being up at all. a queued retry and an honest message is the difference between a delayed sale and a lost one. did you build the retry queue yourself or does your cart handle it, mine just hard fails right now
The thing that saved me once was having the two processors on genuinely separate providers, not two products from the same parent company. The day one went down, the other was on a completely different network and stayed up. If your backup rides the same infrastructure as the primary, a bad enough outage takes both at once. One processor is zero processors as far as i am concerned.
wiring up a backup processor this weekend after reading this, thanks for the nudge. ive got the single processor and hope setup exactly as you describe. better to test the failover on a quiet tuesday on purpose than find out i dont have one at 3am during an outage
month late but also actually test that the failover triggers, not just that the backup exists. i once had a second processor configured and found during a real outage that the health check never fired because it was watching the wrong endpoint. the backup was fine, the switch just never happened. drill the trigger, not only the backup.
the bit people forget is the failed attempts during the outage cost more than the one lost sale. some of them retried on a card and got flagged, so you eat a little fraud score damage too. a clean queue and retry like hank said protects more than the sale, it keeps the attempt tidy.
from the processor side, ask yours if they have a documented status feed you can actually consume, not just a page a human reads. automating failover like owen said only works if you have a reliable signal the primary is really down and not just slow. a lot of bad failovers fire on a blip.
that is a good caution, i was picturing an instant flip and youre right that a twitchy health check flapping between two processors is its own mess. sounds like the trigger needs a bit of patience built in, not just first error then switch.
Two processors is right but make sure they are genuinely independent accounts, ideally at different banks, and not one provider reselling the same acquirer. In high risk especially they can both sit on the same underlying rail, and the day it goes down both of your gateways go dark together and the redundancy was imaginary.