Problem solving as a learnable skill

Here's a little story from my "smart client decisions" honor roll:

This client is making great use of CiviCRM and his organization’s CMS on a regular basis. He's not a software developer, but he does make sure he's familiar with the features that are available to him through the CRM, the CMS, and the community provided extensions, of which he uses many.

He doesn't only reach out to me for emergencies. He'll often check in with me early in the process of planning a new project, to be sure he's on the right track with a good strategy.

But yesterday he reached out with an urgent task.

The problem:

In the midst of a large and important event registration campaign, many registrants — but not all — were getting a “white screen” fatal error.

The credit card payments went through, but they never saw a completion message. Just a white page that said, in effect, "Something went wrong. Please try again."

So they would try again, getting the same message each time.

Meanwhile, their credit card was being charged on each attempt. And they'd still not have confidence that they were registered.

So organization staff were getting dozens of requests for help, and complaints about multiple charges, and an ongoing uncertainty about who's actually registered.

Getting it fixed:

My client did a great job of trying to debug this himself, carefully reproducing the bad behavior with a known set of steps. But as well versed and competent as he is, hes still could not find the source of the problem.

So he reached out to me with an urgent request. He described the problem clearly, and told me how to reproduce it. He told me the solutions he had tried. He told me this was an urgent matter, and explained the business impact.

I took it from there.

I cloned the site offline for in-depth debugging. I discovered that the bad behavior was caused by a bug in a particular CMS extension. I patched the bug on his live site, and tested to be sure it was fixed. I also reported the bug for the extension author, sharing my fix. And I let my client know the problem was resolved.

They're now back up and running, with confidence that people are actually able to register without error.

Lessons:

Why is this on the honor roll? Because my client did several things right:

• As a general practice, he keeps abreast of what's generally possible and what's not.

• He's hands-on managing the site himself, and therefore well aware of how the side is configured and what extensions it relies on.

• He took steps to debug the problem himself first. This way if he fixes it, he retains the knowledge of the solution and lets it inform his configuration work going forward.

• He was methodical in reproducing the bad behavior, and in documenting the steps to do so. This gives him a clear indication of where the problem may lie, as well as a test case he can use to verify the effects of any corrective action.

• He noted the business impact of the problem. This allows him to assess both the urgency and the business value of getting a solution.

• He shared all this with me very clearly so that I could take action without ambiguity.

• Most of all, because of all of these things, he was able to make an informed decision about the cost/benefit calculation to determine whether it was worth pulling in an outside expert.

Overall, this reads like an absolute winner to me.

Here's the thing:

Naturally we all hope that nothing ever goes wrong. Naturally we look for software systems that are well designed and robust.

But no matter what system you're using, surprises will happen.

Dealing with them effectively is a matter of diligence, careful analysis, and smart business driven cost/benefit calculations.

These are all learnable skills. Practice, coaching, and care will move you in the right direction.

All the best,
A.