Implementing A/B (Variant) Testing in Joomla
In running a web site or web business, we always strive to get more traffic from search engines, more social media traffic, more click-throughs and ultimately, more sales. We spend a lot of time on search engine marketing (SEM), search engine optimization (SEO), social media marketing (SMM) and a whole host of efforts to improve sales. Smart web masters are currently spending a lot of time on “responsive design” to improve site usability on mobile devices and tablets. In all of this, we sometimes forget about what happens once a user visits our site. Can users find what caused them to come to the site in the first place? Can they find related material?
To improve the usability of a site, we first start with something that works for the designer, but does that necessarily work for users? When we get into design arguments in a conference room, does the the best design win, or does the loudest designer win? To make sure that the best design wins, you should look at doing “A/B testing” on your web site.
What is A/B Testing
A/B testing is the process of statistically comparing alternative website designs to see which design results in more revenue, a higher click through rate, a longer time on a page or whatever behavior you want to measure. The formal name is “variant testing,” and although the name A/B testing implies only two alternatives, you can compare several alternatives simultaneously but this takes much more site traffic to get statistically valid results in a reasonable time period. Small design changes can sometimes result in major changes in user behavior–you would be amazed at the difference changing the color on a button or link can sometimes make.
If you are a Joomla web master, there really is not much in the way of information on the whole process of configuring A/B testing. For the entire process, the only real descriptions of the process that I could find were this article by Metrist Partners and the related presentation to the Chicago Joomla User Group by Avery Cohen. I used these articles as the basis for setting up the A/B test that is the basis for this article and for a presentation at the North Dallas Joomla User’s Group. With the exception of the specific plugins, all of this article is appropriate for setting up A/B testing on a Wordpress site.
The Steps in Setting up an A/B Test
How you set up and run an A/B test is dependent upon what web analytics tool you use; this tutorial is based upon using Google Analytics, but the general steps will be similar for other tools:
- Choose and install plugins on your web site
- Experiment management plugins
- Page variant plugins
- Define the goal to be measured. This is usually, but not always one of the following:
- Increasing sales
- Increasing the click through rate
- Increasing the time spent on the page or site.
- Create the variant web pages each with a different URL
- Define the experiment parameters in your web analytics tool
- Define the experiment parameters in an experiment manager plugin on your website. The experiment manager plugin must work with your particular web analytics tool.
- Start the experiment in your web analytics tool.
The article that follows goes through the steps in the process to set up an A/B test on this web site. There is one article that has been well received by the target audience for the site, but which does not have a call to action. I wanted to see what would happen if I added a call to action to the article to see if that would increase the number of readers who go to the contact web page.
Choosing and installing Extensions on Your Joomla Site
There are two types of extensions that you will need to install on your web site:
- Experiment management extensions. These extensions work with your web analytics tool–most often Google Analytics–to run and control the experiment.
- Page alternative (variant) design and control extensions. These extensions help you to create page design alternatives in a way that each alternative design (variant) has a different URL, and preferably a URL that the search engine crawlers will not pick up as a new page.
Both of these types of extensions are hard to find in the Joomla extension listing site; very little shows up under “A/B” or “variant” but the sections that follow discuss some of the ones that I found. I have not used all of the ones that are listed here, and there are probably others that I do not list.
Joomla Experiment Management Extensions for Google Analytics
Finding an extension to manage Google Analytics experiments in Joomla was a challenge, but there are a few alternatives both within the Joomla extensions website and on independent web sites:
- Currently listed in Joomla Extensions
- Basewind Google Website Optimizer (Basic). This is the one that I used. It comes in a free version that supports only one concurrent experiment, and a paid version that allows multiple concurrent experiments. It appears to use the Joomla updater, though the code is quite simple (for the free version) in reality and it probably will not get a lot of updates.
- JS Google Content Experiments for Joomla. I did not use this experiment manager, as it does not use the Joomla Updater.
- From Google search
- Visual Website Optimizer. It is not clear that this is currently supported or even available. I have not used this extension.
- Quirky Fox A/B Test for Joomla. This is a paid extension and appears to have support, but it does not appear to use the Joomla Updater.
Joomla Page Variant Design Extensions
Setting up the page variants is easily the most time-consuming part of running an experiment. The only real criterion is whether or not you can use a different URL for the page variants; there are a lot of ways you can do that in Joomla. Whatever approach that you use should not create a URL that competes with the original URL within the Google, Bing and other search engines. My call to action experiment was intentionally simple as you will see later, so I only used one of these packages (Sourcerer), but I can see future experiments where I would use other packages:
- Sourcerer. This package allows you to include PHP into a page for relatively simple or potentially complex changes. The free version is appropriate for sites that do not allow non-administrator logins. If you allow non-administrator users, you will want the paid version so that you can restrict access to the scripting tags to administrative or other trusted users. Allowing untrusted users to use scripting in content is a huge security exposure.
- Chameleon. This looks like the most flexible of the tools that I investigated, but I have not used it yet.
- MetaMod. This appears to be a very flexible tool, but I have not used it yet.
- Advanced Module Manager. This appears to be a very flexible tool, but I have not used it yet.
Design the Alternative Pages for Your Experiment
Designing the variants can be a major task all by itself, and will ultimately be the topic for several future articles. For this first experiment, I used what is probably the easiest technique and used the Sourcerer extension to embed PHP into the web pages and display a different version of the page depending upon whether or not a particular variable was passed. The simple PHP script that follows tests for variables in the URL and then tests for specific variable names and display one of the two call-to-action alternatives depending upon the variable name–?variant_1 or ?variant_2. The three alternative designs have different versions of a call to action following the article:
- The original version with no call to action: https://www.mooresoftwareservices.com/Loan-Pricing/effective-yield-loan-fee-amortization
- A call to action with photo: https://www.mooresoftwareservices.com/Loan-Pricing/effective-yield-loan-fee-amortization?variant_1
- A call to action without photo: https://www.mooresoftwareservices.com/Loan-Pricing/effective-yield-loan-fee-amortization?variant_2
It is important to note that enabling the free version of Sourcerer on a Joomla site that allows non-admin users to write content presents a significant security exposure as the free version does not give you controls to restrict who can and cannot include scripts in their pages. If you have non-admin users who can create content, you need to upgrade to the paid version of Sourcerer.
<?php
defined( '_JEXEC' ) or die;
jimport( 'joomla.plugin.plugin' );
#
$jinput = JFactory::getApplication()->input;
$url = JUri::getInstance();
#print $url . "\n";
#print_r($url->getQuery(true));
#print "\n";
$num_vars = count($url->getQuery(true));
#print $num_vars . "\n";
if ($num_vars == 1) {
if (strcmp(key($url->getQuery(true)),"variant_1")==0) {
print '<div itemscope itemtype="http://schema.org/Organization"> ';
print '<h4>For Assistance</h4>\n';
print '<p>If you have questions on how to implement a fee amortization or other loan-pricing related data warehousing questions, contact Bruce Moore via one of the methods on the <a href="/contact">contact</a> page.';
print '</div>';
print '<img src="/images/bruce_moore_portrait_small.jpg" alt="Portrait of Bruce Moore, owner of Moore Software Services LLC.">';
}
#
# Check for the URL of the second variant for the experiment
#
if (strcmp(key($url->getQuery(true)),"variant_2")==0) {
print '<div itemscope itemtype="http://schema.org/Organization"> ';
print '<h4>For Assistance</h4>\n';
print '<p>If you have questions on how to implement a fee amortization or other loan-pricing related data warehousing questions, contact Bruce Moore via one of the methods on the <a href="/contact">contact</a> page.';
print '</div>';
}
}
?>
Add the code to include the PHP at the bottom of the article
The final step in coding the alternative pages for the Sourcerer/PHP approach is to include the PHP code in the original article using the Sourcerer markup:
{source}
<?php
include $_SERVER['DOCUMENT_ROOT']."experiment_php/mss_google_analytics_ab_testing_call_to_action_1.php";
?>
{/source}
Step 1–Define a Goal in Google Analytics
The first step in setting up A/B variant testing is to define a “goal” in Google Analytics. Usually, the goal is higher sales or a higher click-through rate. Figure 1 shows the admin panel in Google Analytics with the “Goals” option in the lower right column.
After selecting the Goals option as shown in Figure 1, select the red “New Goal” button as shown in Figure 2.
For your new goal, define whether the goal page is a particular URL–the “equals to” option shown in Figure 3–or a family of URLs using the “begins with” option shown in Figure 4.
You should disable the Google Analytics Opt-out extension in your browser while you are defining and testing your configuration.
Step 2–Define an Experiment in Google Analytics
Once you have defined your goal in Google Analytics, you can start the process of defining your experiment. To begin this, go to the Behavior->Experiments left-bar menu option. This will bring up the panel shown in Figure 6. Select “Create experiment” to begin defining your experiment.
Define your experiment under Behavior->Experiments
Name your experiment and set the goal to be measured
Next, as shown in the panel in Figure 7, name your experiment, select the goal to use as the objective to be measured, and decide what percentage of your site’s traffic to use for the experiment. If you are experimenting with an important page that generates a lot of revenue, you probably do not want to put a large percentage of your traffic through the experiment, because the alternative may not perform as well as the original and you do not want to create a financial disaster. On the flip side, if you choose a very low value for a page that does not get a lot of traffic, it will take Google Analytics a long, long time to get statistically valid results.
Step 3–Define Page Variants in Google Analytics
Once you’ve got your experiment named and a goal selected, you must define the URLs for the original version of the page and variant versions of the page.
Set up the original version of the page
For this experiment, the value for the original page in the box for “Web page to experiment” is the URL for the Effective Interest (Yield) Loan Fee Amortization page on this site as shown in Figure 8.
Add URLs for variants of the page
You must add at least one variant for your experiment, in this case a call to action with a photo where ?variant_1 is added to the URL as shown in Figure 9. There are many ways that you can use other URLs depending upon the details of how you constructed the alternative web pages with Joomla.
You can add more than one alternative design
You can add several alternative designs, but the experiment will take more time to collect enough statistics to determine a winner. Figure 10 shows the third alternative (second variant) design where a call to action is included but no photo.
Step 4–Copy Experiment Code from Google Analytics to Basewind Plugin
Once you have defined the URLS for each alternative page design, Google Analytics will give you a short segment of Javascript that must be included in the HTTP header of the original version of the page as shown in Figure 11. This is what the Basewind or other experiment manager extension does for you. Once the Basewind (or other experiment manager extension) is installed you will copy this Javascript into a field in the Basewind plugin (see Figure 14).
Install the Basewind Conversion Optimizer Basic Extension
Before we get to the step for pasting the experiment Javascript, we have to make sure that the Basewind plugin is correctly configured. First, go to Extensions->Plugins and find the Basewind plugin as shown in Figure 12. Select the plugin and change the load ordering so that the Basewind plugin loads last as shown in Figure 13.
Paste the Google Analytics Javascript into the Basewind Plugin
After setting the load order on the first tab of the Basewind plugin configuration, go to the second tab. On the second tab,cd paste the Google Analytics experiment code into the Basewind plugin as shown in Figure 14.
Step 5–Add the Basewind Experiment Tag to the Article
Once you have copied the Google Analyics experiment Javascript to the Basewind plugin, move to the last configuration tab of the Basewind plugin and set the “tag” for the experiment. This tag does not correspond to a Joomla tag nor does it correspond to a Google Tag Manager tag. This tag is used only by the Basewind plugin, and other experiment manager plugins may use a completely different approach.
Set the tag to something that describes the experiment. It should be something that makes it easy to tie back to the Google Analytics experiment name. The free version of the plugin only allows one tag at time, but the paid version allows multiple tags (and corresponding experiment Javascript). The tag is used in the article to tell Basewind to put the experiment Javascript in the HTTP header. Figure 15 shows the tag used for this experiment.
When you have the tag for your experiment set, open up the article (or module where you can enter HTML depending upon your alternative designs) and put the following markup somewhere in the article.
{calltoaction}
This tells the Basewind plugin to add the Google Analytics experiment Javascript to the HTTP header. It does not matter where you put this markup; I put it adjacent to the Sourcerer markup that includes the PHP code for the alternatives as a way to keep track of it.
Step 6–Validate Your Experiment
The final step in the process is to go back to Google Analytics and validate your experiment; Figure 16 shows what it will look like for a successful validation:
- Both experiment and normal Google Analytics code found on the original page
- Only normal Google Analytics code found on each of the variants.
There are several things to check if you have problems validating the experiment
If you have problems validating the experiment, there are several common configuration problems that you should check:
- Make sure that Google Analytics Opt-out Chrome plug in is disabled. I normally have the Google Analytics opt-out plugin enabled in all of my browsers so that my development and testing does not show up as traffic on my web site. This can cause problems during the validation step in the process.
- Check caching and optimization settings in Joomla; you will at least need to clear cache.
- Make sure that Basewind tag is present in the original version of the page.
- Make sure that Basewind loads last.
Summary
Setting up an A/B test for your Joomla web site is really pretty easy, but setting up the page design alternatives can be quite time consuming. The simple Sourcerer/PHP approach used for this example will only work for some types of design alternatives. Other approaches to coding design alternatives will be topics for future articles.