Plan ahead for good reporting

Now and then a client will ask for help reporting on some metrics from their historical data.

Problem is, it can be difficult to get that data because it was not well tracked at the time.

For example, say you need to report accurate numbers on the question, "In each of our training events in the past 2 years, how many attendees were school district employees at the time?"

It's not a bad question, if you have a goal to get school district employees into your trainings.

But it's almost impossible to determine retroactively.

Here's the thing:

If your strategic plan calls for hitting certain metrics in the next year, or 3 years, now is the time to think about how you will report on those numbers.

With a little forethought, you can be well placed for good reporting.

Without it, you might just never know.

All the best,
A.

Good uses for a hammer

Imagine picking up a hammer and asking yourself, "How can I use this hammer around the house?"

You could, after all, use it to open a locked door; or plant tulip bulbs in your garden; or remove a screw.

But why start with the hammer?

Why not ask, "What's the best way to open this locked door? Or plant these tulip bulbs? Or remove this screw?"

You'll probably get better results that way.

All the best,
A.

Start with the problem, not the solution

This morning a client asked me about using tags in CiviCRM.

Essentially the question was, "How could we use tags to track metrics related to our strategic plan?"

It's a fair question, so we discussed it, but soon we decided to zoom out a little and look at things a different way.

I suggested: What if we start by looking at the metrics you'll need to report on? Depending on how those are measured, we can find the best way to track the data, whether it's with tags or something else.

And with that small shift in perspective, we started to home in on the tracking mechanisms that would actually meet her needs.

Spoiler: It's starting to look like tags won't be a big part of her solution.

Here's the thing:

Starting with the tool, or the solution, is one way to go.

But you'll usually get better results by starting with the actual problem, and then selecting the solution that will best solve that problem.

All the best,
A.

Worth the risk?

Witnessed this morning on my way to work:

A truck driver parked his big old semi in the middle of the street, blocking an entire lane, while he walked across the street to Jack In The Box, got his to-go breakfast, and walked back again to drive away.

This was on a major thoroughfare here in town.

Parking ticket? Hassle from the cops? Getting written up by his manager? Having his truck damaged by some driver who was on their phone? Getting yelled at or maybe even pushed around by some hot-head with a penchant for road rage?

These are all concerns he had to weigh, against the value of a breakfast sandwich and a cup of coffee.

Who's to say whether that prize is worth all of those risks?

Only him.

Here's the thing:

We all take chances, everyday, one way or another.

In your personal life, and in your work.

Nothing is guaranteed.

Who gets to decide whether the potential upside is worth the potential risk?

Only you.

All the best,
A.

I’m so fancy

As I write this, I'm sitting in the outdoor lounge at my co-working space.

Down the sidewalk comes a man pulling a hand cart containing one push mower, one weed-eater, and several gas cans. He has a t-shirt that says “Bill's Yard Work.”

While he is still walking by, he’s passed by a large pickup truck hauling a trailer with two zero-turn mowers and a wide assortment of other equipment. The truck is wrapped in a custom paint job reading, “Garcia's Outdoor Services.”

The truck is quickly out of sight. The man just keeps walking.

Observation: These are two businesses in the same line of work, and I have no idea which one is “more successful.”

The Garcia crew looks fancy, but Bill, with his hand cart, seems happy enough.

• Which of them is making a greater profit?

• Which of them takes greater joy in their work?

• Where will either of them be, in terms of what matters to them, in a year, or two?

Here's the thing:

Unless your mission is "looking fancy" — and I’m guessing it’s not — what matters is not how slick your tools look, but how well they help you reach the goals that matter to you.

All the best,
A.

Directions

In my small town, anyone can tell you the best place to get a cup of coffee.

Head down main street. It's the cafe on the right, after you pass the cigar store.

Those directions work great. Every time. Unless:

• You're coming from the less-traveled other end of town.

• You've already tried that cafe and don't like their coffee.

• You're in a hurry and just want any coffee.

Here’s the thing:

Good advice is based on a clear understanding of your starting point, your desired outcomes, and your available resources.

That's true whether it's about coffee, or systems to support your organization's mission.

All the best,
A.

Complex or simple: your choice

I have clients whose CRM system is pretty much on autopilot, and I hear from them once a year or even less, because it's just working for them.

I have other clients who are constantly trying to improve their systems, their programs, their team workflows, and their membership offerings; and for them it's a constant but worthwhile effort to keep their system configured according to their needs.

In between, I have clients who are constantly improving (i.e., changing) their policies and procedures, but who still assume that their robust and flexible CRM system should "just work" for them and they don’t need to think about it much. These folks are struggling, to one degree or another, because their experience hasn't matched their expectations.

Here's the thing:

A simple tool is easy to use but does only a few things very well.

A complex tool can be made to do many things well, but requires more effort to master.

CiviCRM can be either of those things. But, like most things, it can't be both at the same time.

Complexity and simplicity each have their place.

If you know which one you're aiming for, you can usually appreciate both the benefits and the limitations.

But trying to have them both is not usually a recipe for success.

All the best,
A.

Piles

Believe it or not, piles are a fabulously useful organization method. You probably use them a lot.

• Sorting laundry

• Moving into a new house

• Todo lists

• Separating recyclables

They’re not great for the long term, but are wonderfully useful in the short term.

And here's the thing:

A system doesn't have to be perfect for everything.

It just needs to be good enough for the results you need.

Making it any more robust than it needs to be, doesn’t really make it "better".

But it does make it harder, and for no good reason.

All the best,
A.

Branding your CRM

Sometimes inspiring trust means shielding your team from too many technical details.

They just need to believe — with good reason — that their systems are sound and will help them in their work.

I have a couple of clients who take this axiom pretty seriously, even to the point of not naming the underlying CRM software.

To their staff, it’s not “CiviCRM”, it’s just “the CRM” or “our CRM”.

I won’t say that you must do this, but I do agree that it removes needless technical detail from the list of things that compete for your staff users’ attention.

That’s a good thing. It allow them to remain focused on things that matter.

It’s not so hard to do, either:

1. Decide on a name your CRM. You can call it simply “CRM”, or come up with something unique to your organization like “XYZ Association CRM”.

2. Be consistent in referring to your CRM by this name: in emails, in conversation, in training.

3. Use CiviCRM’s own “Word Replacments” feature to replace the word “CiviCRM” with your own CRM name; this replacement will be applied everywhere within CiviCRM, so even CiviCRM will refer to itself by the name you’ve chosen for it.

4. Edit labels for CMS menus that link to CiviCRM, so that they refer to your chosen CRM name instead.

It’s a small step you can take to help your staff users be less concerned about tools, so they can be more focused on outcomes for your mission.

All the best,
A.

Trust issues

If I can convince you that your car's brakes are unreliable, you won't be very likely to drive it unless absolutely necessary.

If your home security system seems to misbehave in ways that you can't understand, you may not sleep so well at night.

That's because even the best system in the world will not be well used if the people who rely on it do not trust it.

This means that giving your staff a great CRM is not, all by itself, sufficient for them to work efficiently.

They must also develop a natural trust in the features of that system and the data they get from it.

So let’s ask:

What steps are you taking to inspire confidence in your systems among your staff?

What steps are you taking to ensure that you yourself can have an appropriate level of confidence in that system?

All the best,
A.

CiviCRM Doc Bot

Want some quick help with CiviCRM but don’t feel like poring over the documentation?

The CiviCRM Doc Bot is an AI assistant that’s been trained on the official CiviCRM documentation. It’s not too shabby, as AI bots go.

Give it a try at its live page here, or install the extension in your own CiviCRM instance (as usual: Administer > System Settings > Extensions >> Add New >> “Doc Bot”) for easy access via a dashlet in your CiviCRM dashboard.

I asked the CiviCRM Doc Bot: “Should I designate an in-house expert for CiviCRM?”, and it said:

Based on the information provided in the CiviCRM documentation, it can be beneficial to designate an in-house CiviCRM expert for your organization. Here are some key points:

• Familiarity with CiviCRM: The documentation states that "a coworker or consultant might be a place to start" if you are not already familiar with using CiviCRM. Having someone in-house who is experienced with the system can help onboard and train other staff.

• Configuration and Customization: The documentation mentions that after installing CiviCRM, you should review the initial configuration tasks to customize it for your organization. An in-house expert can take the lead on this setup and ongoing maintenance.

• Troubleshooting and Support: The documentation suggests talking to a CiviCRM consultant if you run into issues. Having someone internally who can provide first-line support and troubleshoot problems can be very helpful.

• Community Engagement: The documentation encourages users to be active in the CiviCRM community, such as on the CiviCRM StackExchange. An in-house expert can help facilitate this engagement.

The documentation states that "many organizations employ the services of a CiviCRM expert service provider" for tasks like upgrades. However, having someone internally who is the designated CiviCRM expert can complement this and provide ongoing support.

So in summary, the CiviCRM documentation suggests that designating an in-house CiviCRM expert is a good practice that can benefit your organization. This person can help with configuration, training, troubleshooting, and community engagement.

Like I said, not too shabby.

All the best,
A.

CiviCRM upgrade oddities: Joomla wierdness

If you’re not running CiviCRM under Joomla, you can skip this email.

But if you are, you should be aware of an oddity in CiviCRM upgrades that happens specifically under Joomla:

Sometimes the upgrade should have removed some old files, but it does not. This can lead to serious problems:

• An upgrade that can’t complete, requiring you to revert to backups.

• Unpredictable errors such as “file not found” and “Call to undefined method”.

Why is this an upgrade oddity?

The fix for this involves some tasks that non-technical site admins would probably want to avoid, such as one of these:

• Uninstalling and reinstalling CiviCRM. If done correctly, this won’t cause any loss of CiviCRM data, but will require a complete re-doing of your CiviCRM-related user permisisons in Joomla. Yuck.

• Manually identifying (er, how?) the files that should not exist, an then manually deleting them from your server. Double yuck.

Does this problem happen on every upgrade?

No, sometimes you’ll get lucky. But sometimes you won’t. And you won’t know until it happens.

It happens enough that it’s mentioned in the CiviCRM “Upgrade in Joomla” Documention:

… files from previous releases that are no longer required are not removed. Generally this is not a problem but it can cause failures. If this happens you can either … [essentially, the solutions I’ve mentioned above].

Is there an easy way around this?

Not really. You won’t know about it until it happens, and even then the solution is not always obvious. The “Upgrade in Joomla” docs encourage you to “Check on StackExhange or Chat to see if others are reporting the same problem,” which I think no-one would describe as “an easy way around this.”

What if this happens during your upgrade?

If you’re not a developer (and don’t want to become one), your best bet is to revert to your pre-upgrade backup and call in some professional help. An experienced CiviCRM expert can help you get it sorted out pretty easily, but it’s not a battle you’ll want to take on alone.

It’s something to keep in mind.

If your upgrade goes well, and your immediate post-upgrade testing goes well, then you can be reasonably confident that you’ve avoided this problem.

But if it does come up, be glad you have a designated in-house CiviCRM expert and a connection with an outside expert who can help you sort it out. (You do have both of those, right?)

All the best,
A.

CiviCRM upgrade oddities: kooky extra steps

Upgrading CiviCRM can, sometimes, require some unusual and very technical extra steps.

Why is thie an upgrade oddity?

These extra steps are often quite technical in nature, such as performing SQL queries directly in MySQL, editing of PHP files, or other tasks typically handled by a developer.

Non-technical site administrators are not likely to be comfortable with these changes.

But you probbaly won’t even be aware of them until you're in the middle of an upgrade — which can leave you felling a little "stuck".

Are these really so hard to handle?

Depends on your skills, but yes they can be. For example, messages in recent CiviCRM upgrades have included instructions like these:

ACTION REQUIRED: You will need to apply a manual update because your civicrm_activity table is large and the update will run slowly. Please read about how to apply this update manually (linked article inludes specific SQL queries to run in the CivCRM MySQL database).

Your civicrm.settings.php file contains a line to set the php variable `auto_detect_line_endings`. It is deprecated and the line should be removed from the file.

When loading extensions, CiviCRM searches the filesystem. In v5.56+, the default search is more constrained. This improves performance, but some extensions ([insert list here]) will become invisible. To make these visible, you should either move the source-code or edit civicrm.settings.php. For example, you may add this line [insert some PHP code for you to edit into the named file].

Would you be 100% confident taking those steps?

How can you deal with this oddity?

You'll see these messages during the in-browser "Upgrade the database" step of the upgrade process, either at the very beginning or the very end (and sometimes both).

If you don't understand any of the messages, or don't feel confident following the instructions, here's what to do:

Revert to your pre-upgrade backup, and contact your CiviCRM coach or another CiviCRM specialist.

You really can't just ignore these messages?

No, don't even try.

These extra instructions are there for a reason. Ignoring them could leave your system in a broken state — even if it appears to be working properly.

That means you could be moving forward with a CRM that has real problems under the hood, which could be all-but-impossible to resolve later.

No fun. So be careful.

So there you have it. One more reason to be sure you've got good backups before you begin your CiviCRM upgrade.

All the best,
A.

Trade-offs: speed vs accuracy

We can probably agree that speed and accuracy are both desirable.

Do we also agree that there are trade-offs there?

• If you show up late for your final exam, you might have to sacrifice accuracy and hope to get most of the answers right.

• if you're launching a rocket to Mars, it might be worth taking some extra time to double check your calculations.

A CiviCRM example:

This week a client noticed that some of their newsletter recipients received two copies of the mailing.

It happened because of a trade-off between email processing speed and accuracy.

CiviCRM has a setting in the mailer component, "Database Update Frequency."

• A higher value in this setting will help high-volume senders to get their mailings out more quickly.

• But that increase in speed can lead to a decrease in accuracy: some recipients might get duplicate email messages. (This may sound like an odd trade-off but there are some very good technical reasons for this.)

Once we identifid the problem, it was simple enough to recognize that their relatively low sending volume does not demand the increased speed and efficiency.

So we changed that setting to the lowest possible number.

As a result, their outbound mass mailings are now just a little bit slower, but much more accurate.

Here's the thing:

In an ideal world you can have your cake and eat it too.

But in the real world, you're always making trade-offs.

It's good to know what those trade offs are, so you can pick the one that suits you best.

All the best,
A.

CiviCRM upgrade oddities: message template mods

You’re probably familiar with Message Templates in CiviCRM. They’re CiviCRM’s way of providing you with customizable and re-usable content to place into emails such as contribution receipts, event confirmations, and more.

What you might not know is that customizing these templates — while very useful for your communications strategy — can create a nasty headache when upgrading CiviCRM.

The reason is very similar to the problem with file overrides:

You may customize a message template from CiviCRM version A, but if CiviCRM version B includes changes to that message template, you’ll have a tough time getting both the latest changes and your own customizations into one cohesive template.

Why is this an upgrade oddity?

Non-technical site administrators aren’t likely to enjoy this highly technical task, but it’s a required part of some upgrades — if the upgrade includes changes to any message templates that you’ve customized.

Can you just skip this?

You might get away with it, but I don’t recommend it. Message Templates represent important content that your constituents will see.

If you care about your communications strategy, you’ll want to ensure your Message Templates are in good shape.

How can you handle this oddity?

Here are some tips for minimizing the headaches around this oddity:

• Document your customizations. When you’re eding a Message Template, you can add a comment directly in the Message Template like so:
  {* I’m changing X, Y, and Z because of A, B, and C *}
  and that comment won’t be seen by your users. But you (and future site admins) will see it when it’s time to clean up the Message Template.

• Use the excellent extension Better Message Templates. This beautiful add-on will allow you to see exactly how your customized Message Templates differ from the original version provided by CiviCRM — very useful if you never bothered to document your customizations in the first place.

• Take note of your customizations before an upgrade. After the upgrade, CiviCRM may still remind you that some Message Templates need your attention, but by then it can be hard to figure out what your customizations were, and why you made them.

If that’s all too much for you, then do your best to avoid customizing your Message Templates. Just use what CiviCRM gives you out of the box.

Customization is fun. But supporting it properly is work — especially at upgrade time.

All the best,
A.

CiviCRM upgrade oddities: extension compatibility

CiviCRM upgrades should be simple, but they can include some surprises.

For example:

Are you sure that all of your installed extensions are fully compatible the upgraded CiviCRM version?

Extensions can include PHP and Smarty template file overrides, which can create surprising misbehaviors.

They can also rely on CiviCRM core code that isn't available — or that functions differently — in the newest version.

It can be hard to know ahead of time whether this will be an issue.

The solution? Make sure you're testing afterward.

Know what extensions you're using, and what you're using them for. Then after the upgrade, go back and double-check to be sure those things are behaving properly.

All the best,
A.

What's wrong with file overrides?

An astute reader of yesterday’s email pointed out that I didn't actually mention why file overrides are such a problem for upgrades in CiviCRM.

Why can't we just leave the files in place? They worked in the previous version of the CiviCRM, why wouldn't they work in the new version?

The simple answer:

These files don't just contain your customizations; they contain essential pieces of CiviCRM functionality.

That means old versions of these files are very likely incompatible with the new version of CiviCRM.

And if they're incompatible, you will have problems:

• You may be missing out on important security fixes.

• You may be missing out an important bug fixes.

• You may actually be introducing new bugs due to the compatibility.

• Even worse, things may appear to be working fine, while your upgraded system is slowly (and quietly) accumulating corrupt data that you won't notice for months.

Digging out of those problems later on is going to be very difficult or perhaps impossible.

That's why a successful upgrade absolutely depends upon upgrading these files.

So why does CiviCRM allow this at all?

There's nothing inherently wrong with using the file overrides features to customize your system.

But doing so without calculating the cost of increased effort at every single upgrade is probably not a wise move.

All the best,
A.

CiviCRM upgrade oddities: file overrides

One of the best things about CiviCRM is that it's customizable — you can modify it in just about any way you want.

But this great and wonderful thing can make your upgrades awful and painful, if your customizations rely on custom file overrides of PHP or Smarty *.tpl files. This is an officially supported but lightly documented way to customize CiviCRM behavior. It’s falling out of favor for all the reasons I mention here, but it still might affect you.

By the way, this doesn’t even cover any direct modifications of CiviCRM code files that you or other developers may have made on your site. That’s totally unsupported and officially a Bad Idea™.

Why is this an upgrade oddity?

If you have customized files like this, you'll need developer skills to sort it out. This is not a task for a non-technical site owner.

So can you skip this step?

No, unfortunately.

How can you know if this affects you?

It's pretty easy to know, with these steps:

• In CiviCRM, navigate to Administer > System Settings > Directories. This takes you to the Upload Directories page

• Notice these two settings:

  • Custom Templates Directory

  • Custom PHP Directory

• If both of those are blank, you can stop here. This issue doesn't affect you.

• But if there's a value in either of those, more the directory that each one is set to.

• In your hosting system file browser, navigate to those directories and examine their contents.

• If the directories are empty, to can stop here. This issue doesn't affect you.

• But if there are files in either of these directories, this needs your attention. somebody with developer skills needs to examine them and ensure they stay compatible with the upgraded CiviCRM version.

“This sucks. How can I just avoid this?”

Yeah, it’s a mess.

If you're not already in this situation, yes you can avoid it. Just don't use file overrides. And if you hire a developer for custom features, insist that they should not use file overrides either.

If you're already in this situation, there's probably a way to remove those file overrides completely. Whatever customizations they're providing, there probably another way to do it - without file overrides.

A good CiviCRM developer should be able to help you with that. You should probably explain that you're trying to keep your customizations while making upgrades as easy as possible.

Your pre-upgrade homework:

Before your next upgrade, follow the steps above to see if this issue affects you. If it does, ask your CiviCRM coach or a developer for help.

You'll want to get this sorted out before it's time to upgrade.

All the best,
A.

CiviCRM upgrade oddities: backups

One of the first and most important steps in a CiviCRM upgrade is to make sure you have a backup of your entire site.

Why is this an oddity?

This can be a fairly technical task for a non-technical site owner to undertake.

And there is no “one right way” for all CiviCRM sites. The steps to back up your site will vary greatly depending on your hosting platform, your CMS, and your database configuration.

If you've never done it before, you'll need the answers to several important questions first.

So can you skip this step?

No. Absolutely not.

Any upgrade can go wrong, CiviCRM or otherwise. If you don't have a pre-upgrade backup that you can revert to, you may be left with a permanently broken system.

Take the backup.

So how can you handle this oddity?

First, you'll need to answer all of these questions for yourself:

1. Which mySQL database is storing your CMS data?

2. Which mySQL database is storing your CiviCRM data?

3. Where are your site files stored (i.e., on your hosting platform, which directory has your CMS index.php file)?

4. Where are your backups going to be stored?

Once you have those answers, you'll need to select or formulate a process for creating and restoring backups.

CMS-based backup and restore options:

Drupal, WordPress, and other CMSs offer extensions that can back up your site and then restore those backups if needed.

But those methods, by default, won't include your CiviCRM database in the backup. You'll need to see whether they can be configured to do that.

Hosting-based backup options:

Many hosting providers do offer control-panel-based tools to backup your site, and to restore the site from a backup.

Check with your hosting provider to see what your options are.

Testing your backups:

A backup that's incomplete is about as good as no backup at all.

You'll want to test your backup system to ensure it covers everything.

The challenge is this: If you're not certain that your backup is complete (which is why you're testing it), do you really want to test it on your live site?

Probably not.

The answer is this: Whatever backup method you're using (CMS-based, or hosting-based), it should offer some way to restore your backup to a different location, such as your sandbox site.

That way you can take a backup of your live site, restore it into your sandbox site, and peruse the sandbox site to be sure the backup and restore both worked well.

Getting help:

Pre-upgrade backups are not something you want to skip. Getting this wrong can ruin years of priceless CRM data.

If you can't figure out on your own how to backup and restore all databases and files, get some help.

A developer or a CiviCRM coach should be able to help you get this working, so that you have reliable and reusable tools to manage your backups.

Don't take chances with your priceless constituent data.

All the best,
A.

CiviCRM upgrade oddities: the sandbox site

CiviCRM's upgrade documentation instructs you to test your upgrade first on a copy of the live site, and only after that to actually upgrade the live site.

Why is this an oddity?

Your average non-technical site owner will not be able to create a working copy of their live site without professional help.

So is this a step you can skip?

I'll admit, I often skip it. But I've been doing CiviCRM upgrades for 15 years.

For your first several times applying a CiviCRM upgrade, I cannot in good conscience recommend that you skip this step.

If you have a high risk tolerance, you might ignore that advice and skip this step anyway. That's fine. Just don't tell anyone I recommended it.

So how can you handle this oddity?

You have a couple of options:

1. Learn how to create a working copy of your live site. You may find a CiviCRM coach who can teach you how to do it. Or, you may enjoy learning this on your own.

2. Hire a developer to create a tool that will clone your live site to a working “sandbox” copy. This is a software development task, and perfectly suitable for outsourcing (because you don't care how they build it, you just care that it works whenever you need to use it).

Are there other reasons you would want to have a sandbox site?

Yes, there probably are.

A sandbox site is a place where you can test any number of things:

• CiviCRM upgrades

• CMS upgrades

• New configurations and features

• Mew content and designs

• Data imports

• Lots more …

If you try that stuff on your live site and it goes poorly, you may have some big headaches. But if that happens on your sandbox site, you can just throw it all away by rebuilding the sandbox site from the live site.

Your pre-upgrade homework:

If you’re interested in handling CiviCRM upgrades in-house, you’ll need to put some thought into this part of it.

Find out what it would take to build a sandbox copy of your life site. Can you do it yourself? Could you have developer automate it, so you can rebuild the sandbox site whenever you want?

To sum up:

Building a working “sandbox” copy of your live site is something you may never have thought of before. But the CiviCRM documentation recommends it for good reason. It's also nice to have for other things, not just CiviCRM upgrades.

All the best,
A.