Preview CiviCRM’s upgrade alerts

Here’s a nice tool that will help you get ready for your next CiviCRM upgrade: CiviCRM Upgrade Message Previewer.

How does this make upgrades easier?

Upgrades in CiviCRM will sometimes present you with alert messages that need a little extra thought — and sometimes need a little extra action.

Normally, the first time you encounter these messages is while you’re in the middle of an upgrade. That is, after you’ve already loaded the newest CiviCRM code files on your site.

Considering that some of these messages call for additional action on your part, it can help a lot to know what to expect ahead of time, so you can be ready.

How does it work?

Just tell it your current CiviCRM version, the version you’ll be upgrading to, and your CMS (Drupal, WordPress, etc.), and click the button.

You’ll get a full list of the pre-upgrade and post-upgrade messages you can expect to see in that upgrade, including helpful descriptions and links to additional documentation when it’s relevant.

Shouldn’t I see these messages when I upgrade my sandbox site?

Yes! If you’re making the extra effort to upgrade a sandbox copy of your live site first — as a test before actually upgrading you’re live site — that’s great! It’s the recommended practice.

And when you do that, you’ll see these same messages. Since the site is just a sandbox copy and not your actual live site, you’ll have time to react to those messages.

But for folks who still have a hard time building a sandbox copy of their site, that kind of prevew workflow is not an easy option.

All the best,
A.

How accurate are mailing statistics?

This (paraphrased) question came in today from a client:

CiviMail is reporting that over 50 recipients clicked the link to register for our event, but this seems very unlikely. How reliable are these numbers?

It’s a smart question. Truth is, the "Click-through" numbers can be artificially high, for reasons beyond your (or your CRM’s) control.

How could this be?

One reason is that the company that hosts your recipients’ email accounts may do things that make it look like the recipient clicked one or more links in the email.

This is true regardless of what system you might use to send the email — CiviCRM, your own gmail account, some other newsletter software, whatever.

Why would they do that?

The usual rationale here is that the email hosting company wants to block email messages that contain links to known fraud or otherwise malicious content.

So, they'll actually open every single link in the email message, before it even gets to the recipient's in-box.

(For example, several of my clients’ recipients appear to have clicked on all 11 links in a given mailing, but we agree that’s just unlikely.)

Can’t CiviCRM figure that out and adjust the statistics?

Unfortunately, there's no easy way to know if the link was "opened" by the actual human recipient or by the email company's scanning software.

This has the effect of artificially inflating the click tracking in your mailing reports.

FWIW, there are some ideas floating around out there "on the web" about how to deal with this kind of problem, but these ideas don't have a lot of traction yet.

Most mass-mailing solutions (including CiviCRM) will need work before they're able to implement such ideas to help get more accurate numbers in the face of this kind of "pre-scanning" activity.

So these numbers are just useless?

Not entirely. Because not all email hosting services are doing this kind of thing, the click-through rates on your emails are still good as relative indicators:

• if one of your mailings shows a click-through rate that’s relatively lower or higher than your other mailings, that probably indicates that more or fewer real people clicked those links.

• But as an absolute indicator of “who clicked what”, yes, these numbers are less useful than you might have thought.

To sum up:

It's a problem that lots of other people are talking about (not just in CiviCRM) but no one has a solid solution yet.

As a result, meanwhile, you should assume that the click-through numbers are artificially inflated.

To quote one blogger on this topic, "Having misleading statistics is just something we have to deal with as a marketer."

More info:

For reference, here's a (rather long) blog post from civicrm.org that tries to explain all of this. It might be interesting if you have the time.

All the best,
A.

“Ought to”?

“Would you tell me, please, which way I ought to go from here?”

“That depends a good deal on where you want to get to,” said the Cat.

“I don’t much care where—” said Alice.

“Then it doesn’t matter which way you go,” said the Cat.

“—so long as I get somewhere,” Alice added as an explanation.

“Oh, you’re sure to do that,” said the Cat, “if you only walk long enough.”

From Alice’s Adventures in Wonderland, by Lewis Carroll

Here’s the thing:

No one can tell you which way you ought to go. It depends a good deal on where you want to get to.

All the best,
A.

Plan ahead for good reporting

Now and then a client will ask for help reporting on some metrics from their historical data.

Problem is, it can be difficult to get that data because it was not well tracked at the time.

For example, say you need to report accurate numbers on the question, "In each of our training events in the past 2 years, how many attendees were school district employees at the time?"

It's not a bad question, if you have a goal to get school district employees into your trainings.

But it's almost impossible to determine retroactively.

Here's the thing:

If your strategic plan calls for hitting certain metrics in the next year, or 3 years, now is the time to think about how you will report on those numbers.

With a little forethought, you can be well placed for good reporting.

Without it, you might just never know.

All the best,
A.

Good uses for a hammer

Imagine picking up a hammer and asking yourself, "How can I use this hammer around the house?"

You could, after all, use it to open a locked door; or plant tulip bulbs in your garden; or remove a screw.

But why start with the hammer?

Why not ask, "What's the best way to open this locked door? Or plant these tulip bulbs? Or remove this screw?"

You'll probably get better results that way.

All the best,
A.

Start with the problem, not the solution

This morning a client asked me about using tags in CiviCRM.

Essentially the question was, "How could we use tags to track metrics related to our strategic plan?"

It's a fair question, so we discussed it, but soon we decided to zoom out a little and look at things a different way.

I suggested: What if we start by looking at the metrics you'll need to report on? Depending on how those are measured, we can find the best way to track the data, whether it's with tags or something else.

And with that small shift in perspective, we started to home in on the tracking mechanisms that would actually meet her needs.

Spoiler: It's starting to look like tags won't be a big part of her solution.

Here's the thing:

Starting with the tool, or the solution, is one way to go.

But you'll usually get better results by starting with the actual problem, and then selecting the solution that will best solve that problem.

All the best,
A.

Worth the risk?

Witnessed this morning on my way to work:

A truck driver parked his big old semi in the middle of the street, blocking an entire lane, while he walked across the street to Jack In The Box, got his to-go breakfast, and walked back again to drive away.

This was on a major thoroughfare here in town.

Parking ticket? Hassle from the cops? Getting written up by his manager? Having his truck damaged by some driver who was on their phone? Getting yelled at or maybe even pushed around by some hot-head with a penchant for road rage?

These are all concerns he had to weigh, against the value of a breakfast sandwich and a cup of coffee.

Who's to say whether that prize is worth all of those risks?

Only him.

Here's the thing:

We all take chances, everyday, one way or another.

In your personal life, and in your work.

Nothing is guaranteed.

Who gets to decide whether the potential upside is worth the potential risk?

Only you.

All the best,
A.

I’m so fancy

As I write this, I'm sitting in the outdoor lounge at my co-working space.

Down the sidewalk comes a man pulling a hand cart containing one push mower, one weed-eater, and several gas cans. He has a t-shirt that says “Bill's Yard Work.”

While he is still walking by, he’s passed by a large pickup truck hauling a trailer with two zero-turn mowers and a wide assortment of other equipment. The truck is wrapped in a custom paint job reading, “Garcia's Outdoor Services.”

The truck is quickly out of sight. The man just keeps walking.

Observation: These are two businesses in the same line of work, and I have no idea which one is “more successful.”

The Garcia crew looks fancy, but Bill, with his hand cart, seems happy enough.

• Which of them is making a greater profit?

• Which of them takes greater joy in their work?

• Where will either of them be, in terms of what matters to them, in a year, or two?

Here's the thing:

Unless your mission is "looking fancy" — and I’m guessing it’s not — what matters is not how slick your tools look, but how well they help you reach the goals that matter to you.

All the best,
A.

Directions

In my small town, anyone can tell you the best place to get a cup of coffee.

Head down main street. It's the cafe on the right, after you pass the cigar store.

Those directions work great. Every time. Unless:

• You're coming from the less-traveled other end of town.

• You've already tried that cafe and don't like their coffee.

• You're in a hurry and just want any coffee.

Here’s the thing:

Good advice is based on a clear understanding of your starting point, your desired outcomes, and your available resources.

That's true whether it's about coffee, or systems to support your organization's mission.

All the best,
A.

Complex or simple: your choice

I have clients whose CRM system is pretty much on autopilot, and I hear from them once a year or even less, because it's just working for them.

I have other clients who are constantly trying to improve their systems, their programs, their team workflows, and their membership offerings; and for them it's a constant but worthwhile effort to keep their system configured according to their needs.

In between, I have clients who are constantly improving (i.e., changing) their policies and procedures, but who still assume that their robust and flexible CRM system should "just work" for them and they don’t need to think about it much. These folks are struggling, to one degree or another, because their experience hasn't matched their expectations.

Here's the thing:

A simple tool is easy to use but does only a few things very well.

A complex tool can be made to do many things well, but requires more effort to master.

CiviCRM can be either of those things. But, like most things, it can't be both at the same time.

Complexity and simplicity each have their place.

If you know which one you're aiming for, you can usually appreciate both the benefits and the limitations.

But trying to have them both is not usually a recipe for success.

All the best,
A.

Piles

Believe it or not, piles are a fabulously useful organization method. You probably use them a lot.

• Sorting laundry

• Moving into a new house

• Todo lists

• Separating recyclables

They’re not great for the long term, but are wonderfully useful in the short term.

And here's the thing:

A system doesn't have to be perfect for everything.

It just needs to be good enough for the results you need.

Making it any more robust than it needs to be, doesn’t really make it "better".

But it does make it harder, and for no good reason.

All the best,
A.

Branding your CRM

Sometimes inspiring trust means shielding your team from too many technical details.

They just need to believe — with good reason — that their systems are sound and will help them in their work.

I have a couple of clients who take this axiom pretty seriously, even to the point of not naming the underlying CRM software.

To their staff, it’s not “CiviCRM”, it’s just “the CRM” or “our CRM”.

I won’t say that you must do this, but I do agree that it removes needless technical detail from the list of things that compete for your staff users’ attention.

That’s a good thing. It allow them to remain focused on things that matter.

It’s not so hard to do, either:

1. Decide on a name your CRM. You can call it simply “CRM”, or come up with something unique to your organization like “XYZ Association CRM”.

2. Be consistent in referring to your CRM by this name: in emails, in conversation, in training.

3. Use CiviCRM’s own “Word Replacments” feature to replace the word “CiviCRM” with your own CRM name; this replacement will be applied everywhere within CiviCRM, so even CiviCRM will refer to itself by the name you’ve chosen for it.

4. Edit labels for CMS menus that link to CiviCRM, so that they refer to your chosen CRM name instead.

It’s a small step you can take to help your staff users be less concerned about tools, so they can be more focused on outcomes for your mission.

All the best,
A.

Trust issues

If I can convince you that your car's brakes are unreliable, you won't be very likely to drive it unless absolutely necessary.

If your home security system seems to misbehave in ways that you can't understand, you may not sleep so well at night.

That's because even the best system in the world will not be well used if the people who rely on it do not trust it.

This means that giving your staff a great CRM is not, all by itself, sufficient for them to work efficiently.

They must also develop a natural trust in the features of that system and the data they get from it.

So let’s ask:

What steps are you taking to inspire confidence in your systems among your staff?

What steps are you taking to ensure that you yourself can have an appropriate level of confidence in that system?

All the best,
A.

CiviCRM Doc Bot

Want some quick help with CiviCRM but don’t feel like poring over the documentation?

The CiviCRM Doc Bot is an AI assistant that’s been trained on the official CiviCRM documentation. It’s not too shabby, as AI bots go.

Give it a try at its live page here, or install the extension in your own CiviCRM instance (as usual: Administer > System Settings > Extensions >> Add New >> “Doc Bot”) for easy access via a dashlet in your CiviCRM dashboard.

I asked the CiviCRM Doc Bot: “Should I designate an in-house expert for CiviCRM?”, and it said:

Based on the information provided in the CiviCRM documentation, it can be beneficial to designate an in-house CiviCRM expert for your organization. Here are some key points:

• Familiarity with CiviCRM: The documentation states that "a coworker or consultant might be a place to start" if you are not already familiar with using CiviCRM. Having someone in-house who is experienced with the system can help onboard and train other staff.

• Configuration and Customization: The documentation mentions that after installing CiviCRM, you should review the initial configuration tasks to customize it for your organization. An in-house expert can take the lead on this setup and ongoing maintenance.

• Troubleshooting and Support: The documentation suggests talking to a CiviCRM consultant if you run into issues. Having someone internally who can provide first-line support and troubleshoot problems can be very helpful.

• Community Engagement: The documentation encourages users to be active in the CiviCRM community, such as on the CiviCRM StackExchange. An in-house expert can help facilitate this engagement.

The documentation states that "many organizations employ the services of a CiviCRM expert service provider" for tasks like upgrades. However, having someone internally who is the designated CiviCRM expert can complement this and provide ongoing support.

So in summary, the CiviCRM documentation suggests that designating an in-house CiviCRM expert is a good practice that can benefit your organization. This person can help with configuration, training, troubleshooting, and community engagement.

Like I said, not too shabby.

All the best,
A.

CiviCRM upgrade oddities: Joomla wierdness

If you’re not running CiviCRM under Joomla, you can skip this email.

But if you are, you should be aware of an oddity in CiviCRM upgrades that happens specifically under Joomla:

Sometimes the upgrade should have removed some old files, but it does not. This can lead to serious problems:

• An upgrade that can’t complete, requiring you to revert to backups.

• Unpredictable errors such as “file not found” and “Call to undefined method”.

Why is this an upgrade oddity?

The fix for this involves some tasks that non-technical site admins would probably want to avoid, such as one of these:

• Uninstalling and reinstalling CiviCRM. If done correctly, this won’t cause any loss of CiviCRM data, but will require a complete re-doing of your CiviCRM-related user permisisons in Joomla. Yuck.

• Manually identifying (er, how?) the files that should not exist, an then manually deleting them from your server. Double yuck.

Does this problem happen on every upgrade?

No, sometimes you’ll get lucky. But sometimes you won’t. And you won’t know until it happens.

It happens enough that it’s mentioned in the CiviCRM “Upgrade in Joomla” Documention:

… files from previous releases that are no longer required are not removed. Generally this is not a problem but it can cause failures. If this happens you can either … [essentially, the solutions I’ve mentioned above].

Is there an easy way around this?

Not really. You won’t know about it until it happens, and even then the solution is not always obvious. The “Upgrade in Joomla” docs encourage you to “Check on StackExhange or Chat to see if others are reporting the same problem,” which I think no-one would describe as “an easy way around this.”

What if this happens during your upgrade?

If you’re not a developer (and don’t want to become one), your best bet is to revert to your pre-upgrade backup and call in some professional help. An experienced CiviCRM expert can help you get it sorted out pretty easily, but it’s not a battle you’ll want to take on alone.

It’s something to keep in mind.

If your upgrade goes well, and your immediate post-upgrade testing goes well, then you can be reasonably confident that you’ve avoided this problem.

But if it does come up, be glad you have a designated in-house CiviCRM expert and a connection with an outside expert who can help you sort it out. (You do have both of those, right?)

All the best,
A.

CiviCRM upgrade oddities: kooky extra steps

Upgrading CiviCRM can, sometimes, require some unusual and very technical extra steps.

Why is this an upgrade oddity?

These extra steps are often quite technical in nature, such as performing SQL queries directly in MySQL, editing of PHP files, or other tasks typically handled by a developer.

Non-technical site administrators are not likely to be comfortable with these changes.

But you probbaly won’t even be aware of them until you're in the middle of an upgrade — which can leave you felling a little "stuck".

Are these really so hard to handle?

Depends on your skills, but yes they can be. For example, messages in recent CiviCRM upgrades have included instructions like these:

ACTION REQUIRED: You will need to apply a manual update because your civicrm_activity table is large and the update will run slowly. Please read about how to apply this update manually (linked article inludes specific SQL queries to run in the CivCRM MySQL database).

Your civicrm.settings.php file contains a line to set the php variable `auto_detect_line_endings`. It is deprecated and the line should be removed from the file.

When loading extensions, CiviCRM searches the filesystem. In v5.56+, the default search is more constrained. This improves performance, but some extensions ([insert list here]) will become invisible. To make these visible, you should either move the source-code or edit civicrm.settings.php. For example, you may add this line [insert some PHP code for you to edit into the named file].

Would you be 100% confident taking those steps?

How can you deal with this oddity?

You'll see these messages during the in-browser "Upgrade the database" step of the upgrade process, either at the very beginning or the very end (and sometimes both).

If you don't understand any of the messages, or don't feel confident following the instructions, here's what to do:

Revert to your pre-upgrade backup, and contact your CiviCRM coach or another CiviCRM specialist.

You really can't just ignore these messages?

No, don't even try.

These extra instructions are there for a reason. Ignoring them could leave your system in a broken state — even if it appears to be working properly.

That means you could be moving forward with a CRM that has real problems under the hood, which could be all-but-impossible to resolve later.

No fun. So be careful.

So there you have it. One more reason to be sure you've got good backups before you begin your CiviCRM upgrade.

All the best,
A.

Trade-offs: speed vs accuracy

We can probably agree that speed and accuracy are both desirable.

Do we also agree that there are trade-offs there?

• If you show up late for your final exam, you might have to sacrifice accuracy and hope to get most of the answers right.

• if you're launching a rocket to Mars, it might be worth taking some extra time to double check your calculations.

A CiviCRM example:

This week a client noticed that some of their newsletter recipients received two copies of the mailing.

It happened because of a trade-off between email processing speed and accuracy.

CiviCRM has a setting in the mailer component, "Database Update Frequency."

• A higher value in this setting will help high-volume senders to get their mailings out more quickly.

• But that increase in speed can lead to a decrease in accuracy: some recipients might get duplicate email messages. (This may sound like an odd trade-off but there are some very good technical reasons for this.)

Once we identifid the problem, it was simple enough to recognize that their relatively low sending volume does not demand the increased speed and efficiency.

So we changed that setting to the lowest possible number.

As a result, their outbound mass mailings are now just a little bit slower, but much more accurate.

Here's the thing:

In an ideal world you can have your cake and eat it too.

But in the real world, you're always making trade-offs.

It's good to know what those trade offs are, so you can pick the one that suits you best.

All the best,
A.

CiviCRM upgrade oddities: message template mods

You’re probably familiar with Message Templates in CiviCRM. They’re CiviCRM’s way of providing you with customizable and re-usable content to place into emails such as contribution receipts, event confirmations, and more.

What you might not know is that customizing these templates — while very useful for your communications strategy — can create a nasty headache when upgrading CiviCRM.

The reason is very similar to the problem with file overrides:

You may customize a message template from CiviCRM version A, but if CiviCRM version B includes changes to that message template, you’ll have a tough time getting both the latest changes and your own customizations into one cohesive template.

Why is this an upgrade oddity?

Non-technical site administrators aren’t likely to enjoy this highly technical task, but it’s a required part of some upgrades — if the upgrade includes changes to any message templates that you’ve customized.

Can you just skip this?

You might get away with it, but I don’t recommend it. Message Templates represent important content that your constituents will see.

If you care about your communications strategy, you’ll want to ensure your Message Templates are in good shape.

How can you handle this oddity?

Here are some tips for minimizing the headaches around this oddity:

• Document your customizations. When you’re eding a Message Template, you can add a comment directly in the Message Template like so:
  {* I’m changing X, Y, and Z because of A, B, and C *}
  and that comment won’t be seen by your users. But you (and future site admins) will see it when it’s time to clean up the Message Template.

• Use the excellent extension Better Message Templates. This beautiful add-on will allow you to see exactly how your customized Message Templates differ from the original version provided by CiviCRM — very useful if you never bothered to document your customizations in the first place.

• Take note of your customizations before an upgrade. After the upgrade, CiviCRM may still remind you that some Message Templates need your attention, but by then it can be hard to figure out what your customizations were, and why you made them.

If that’s all too much for you, then do your best to avoid customizing your Message Templates. Just use what CiviCRM gives you out of the box.

Customization is fun. But supporting it properly is work — especially at upgrade time.

All the best,
A.

CiviCRM upgrade oddities: extension compatibility

CiviCRM upgrades should be simple, but they can include some surprises.

For example:

Are you sure that all of your installed extensions are fully compatible the upgraded CiviCRM version?

Extensions can include PHP and Smarty template file overrides, which can create surprising misbehaviors.

They can also rely on CiviCRM core code that isn't available — or that functions differently — in the newest version.

It can be hard to know ahead of time whether this will be an issue.

The solution? Make sure you're testing afterward.

Know what extensions you're using, and what you're using them for. Then after the upgrade, go back and double-check to be sure those things are behaving properly.

All the best,
A.

What's wrong with file overrides?

An astute reader of yesterday’s email pointed out that I didn't actually mention why file overrides are such a problem for upgrades in CiviCRM.

Why can't we just leave the files in place? They worked in the previous version of the CiviCRM, why wouldn't they work in the new version?

The simple answer:

These files don't just contain your customizations; they contain essential pieces of CiviCRM functionality.

That means old versions of these files are very likely incompatible with the new version of CiviCRM.

And if they're incompatible, you will have problems:

• You may be missing out on important security fixes.

• You may be missing out an important bug fixes.

• You may actually be introducing new bugs due to the compatibility.

• Even worse, things may appear to be working fine, while your upgraded system is slowly (and quietly) accumulating corrupt data that you won't notice for months.

Digging out of those problems later on is going to be very difficult or perhaps impossible.

That's why a successful upgrade absolutely depends upon upgrading these files.

So why does CiviCRM allow this at all?

There's nothing inherently wrong with using the file overrides features to customize your system.

But doing so without calculating the cost of increased effort at every single upgrade is probably not a wise move.

All the best,
A.