Organizing data is a critical planning step in your transition to CiviCRM. What sort of contacts will you be managing and what are your data needs? How will your contacts be grouped and tagged? What relationships will they have between each other and you? What information will you want to get out of CiviCRM? Once you have a plan in place, you will want to get your existing data into CiviCRM the way you want. This often exposes the poor quality of your data, and the effort required to clean it up and get it into shape. This can sometimes take longer than the data planning stage. This chapter shows you some recipes that don't even use CiviCRM but will help get your data in shape. In other recipes we'll explore some of the techniques of organizing your data once you get it into CiviCRM.

Imagine you are managing a soccer club using CiviCRM. For some contacts—players—you would want to record the goals scored, attendance, age, playing position, injuries, and other data. For other contacts such as coaches, you would want to collect different data: first aid qualifications, coaching qualifications, and so on. Having both these sets of data showing up on every contact summary screen does not make sense. What you need is to have two different sorts of contacts; contacts who are players and contacts who are coaches so that they both hold separate data. CiviCRM makes this easy to accomplish.

Relationships are a central feature of CiviCRM. For example, you may run an organization that has contacts related to it because they are volunteers, or you may have individual contacts that are the children of other contacts who are their parents. Some relationships are time-limited. For example, the chair of an organization may have a one year fixed term of office.

Tags are a way of organizing your contacts, activities, and cases within CiviCRM. You can regard a tag as a way of describing data and as a powerful way to segment your contacts. Any tags you create are visible on the contact edit screen so they are great with getting summary information about a contact quickly. You need to include tags in your data plan for CiviCRM, otherwise your screens can get overrun by redundant tags.

Tags are ways of describing your contacts. When you look at a contact summary screen, all the tags applied to the contact are visible, providing at-a-glance information without having to drill down. Contacts can also be segmented into groups. Groups are not visible on the contact summary screen so they are not useful for getting instant information. They come into their own when they are used for other actions within CiviCRM. For example, they can be used to control permissions, or as mailing lists. Once you get to a group listing there are a wide range of actions you can apply to contacts within the group.

Option lists are used throughout CiviCRM to ensure data integrity and to make CiviCRM core functions work. Some of these are not available from the menu system.

How to do it…

The option lists available through the menu system are limited. There is a main screen where you can edit all option lists:

Here there are only a few drop-down options available to edit.

1. Navigate to Administer | Administration Console.

   

2. In System Settings you will see the Option Groups link in the second column. This provides you with the full list. Many of these option lists are generated internally by CiviCRM so you need to be very careful about which ones you alter.

Searching for contacts and doing something with the set of results is a day-to-day task in CiviCRM. For example, you might want to find all your contacts that live in Alaska and send them a mail shot. This is pretty easy to do in Advanced Search, and you can add all the contacts you find into a group called Alaska. This means that you can always go back to the group and find the contacts you added. The group represents a snapshot of your data that you took when you did the search and added contacts to the group.

The problem is that if new contacts are added into CiviCRM who live in Alaska, you will have to remember to add them to the group, and you will have to remove them from the group if they move from Alaska.

To overcome this problem, CiviCRM has smart groups. Smart groups automatically contain any contacts that match your search criteria. So, if the data in your contacts changes, or contacts are added or removed, this is reflected dynamically in the...

Preparing data for CiviCRM import can be a time-consuming, frustrating, and traumatic experience. But it is a job that has to be done. CiviCRM does an enormous amount of error checking on data import and will not import records with errors it spots.

Consider this data:

Here you can see that there are data inconsistencies in the center column. The town Poulton-le-Fylde has five different ways of spelling and presenting the data. This is quite a common problem in legacy systems that were designed to hold addressing data for label printing rather than for searching or geocoding. Another common problem is having data in the wrong columns. Towns, cities, and postcodes are often spread across many columns. The result is you cannot guarantee accurate search results or do any geocoding.

There are occasions where you want to get some data into CiviCRM but there is no quick way of doing it. For example, your existing contacts may all be tagged. You want to get these tag values into CiviCRM so that when you import your contacts your tags work properly.

The CiviCRM interface only allows you to add one tag at a time. So this could be very time-consuming if you have hundreds of tags. This recipe introduces the use of the command-line interface to rapidly add data to CiviCRM.

The recipe can be used to migrate data into most CiviCRM tables. It's not as terrifying as it sounds.