Archive for 'Voice of Customer'

Recently, I was re-reading one of Avinash Kaushik’s older blog posts on tracking Internal Search Term Exit Rates and realized that I had never discussed how to report on this using Omniture SiteCatalyst.  In a past Internal Search post, I covered many different things you can do to track internal search on your site, but did not cover ways to see which terms are doing well and which are not.  In this post I will share how you can see this so you can determine which search terms need help…

Why Track Internal Search Term Click-Through & Exit Rates?
So why should you do this?  In the era of Google, we are all slowly being trained to find things through search.  Many of my past clients saw the percent of website visitors using search rise over the past few years.  In addition, Internal Search and Voice of Customer tools are some of the few out there where you can see the intent of your visitors.  Unfortunately, most websites have horrible Internal Search results which can lead to site exits.  In my previous Internal Search post I demonstrated how to track your Search Results Page Exit Rate, but that only shows you if you have a problem or not.  If you do have a high Search Results Page Exit Rate, the next logical step is to determine which search terms your users think have relevant search results and which do not.  Note that this is not meant to show you which terms lead directly to website exits, but rather, which terms cause visitors to use or not use the search results you offer them after they search on a particular term.

How Do You Track Internal Search Term Click-Through & Exit Rates?
Ok, so how do you do this?  Follow the following implementation steps:

• Make sure that you are setting a Success Event when visitors conduct Internal Searches on your website.  Hopefully you are already doing this so in many cases this step will be done!
• Make sure that you are capturing the Internal Search Term the visitor searched upon in an eVar variable.  Again, you should be doing this (if not, shame on you!).
• Here is where we get into uncharted territory.  The next step is to set a new Success Event when visitors click on one of the items on the search results page.  Depending upon the technology you use for Internal Search, this could be hard or easy.  Regardless of how you actually code it, the key here is to set the second Success Event (I call it Internal Search Results Clicks) only if visitors click on a search result item (not if they click on a second page of search results or go to another page through other navigation).  It is also critically important that you only set this Search Results Clicks Success Event once per search term!  Do not set it every time a visitor clicks on one of the search results after using the “Back” button.  If you don’t do this correctly, your Click-Through and Exit Rates will be off.  This could take a few iterations to get right, but stick with it!
• Once you have both the Internal Searches and Internal Search Results Clicks Success Events set, you can create a Calculated Metric that divides Internal Search Results Clicks by Internal Searches to see the Internal Search Click-Through Rate as shown here:

• From there you can create the converse metric which subtracts the Internal Search Click Through Rate by “1″ to come up with the Internal Search Exit Rate as shown here:

• After this is done, you can open the Internal Search Term eVar report and add all three metrics so you see a report like this:

In this case, it looks like the “Zune” Internal Search term might need some different search result content as it has a much higher exit rate as the others.  Another cool thing you can do is to create a report which trends the Internal Search CTR % or Exit % for specific Internal Search terms so you can see if they have been good/bad over time.  Also, if you use SAINT Classifications to group your Internal Search Terms into buckets, you can see the report above for groups of Internal Search terms.  If you vote for my idea in the Idea Exchange, you would be able to set SiteCatalyst Alerts to be notified if your top Internal Search Terms have spikes in their Click-Through or Exit Rates.  You can also segment your data to see how the Internal Search rates differ when people come from Paid Search vs. SEO, etc… and even use Test&Target to try out different promotional banners on your search results page…

Finally, don’t forget that when you create a new calculated metric like the Internal Search CTR % metric described above, you also get the bonus of seeing this metric across your entire website under the My Calc Metrics area of the SiteCatalyst toolbar.  Simply find this new metric and click on it and you can see your overall Internal Search Click-Through Rate regardless of internal search term.  Your report will look something like this:

For The True Web Analyst Geeks
If you were bothered when I mentioned above that you should only set the Search Results Clicks Success Event once per search term, then you are my kind of person (please apply for a job with me!)!  You were probably saying to yourself: “If I only count once per search term, how will I know which search terms get visitors to click on multiple search result links?”  Right you are!  That could be valuable information.  If you want to see that as well, all you have to do is set a second Success Event each time a visitor clicks on a search result [I call this Internal Search Result Clicks (All)].  Then you can compare how many times people click on any search result to how often click in total.  Here is a sample report:

In this example, you can see that the search term “api” had one click only in either scenario, but the search term “chatter” had people click on it 100% of the time and 5 times they clicked on two search result items.  If you want, you can create another Calculated Metric that divides the Internal Search Result Clicks (All) by the # of Internal Searches to see how many search result clicks each term averages.  In the case of “chatter” above, it would be 2.25 search result clicks per search!

Final Thoughts
If Internal Search is important to your site, make sure you are tracking it adequately so you can improve it and increase your overall website conversion.  Do you have any other cool Internal Search tracking tips I haven’t covered?  If so, leave a comment here…

In the Web Analytics space, we spend a lot of time recording and analyzing what people do on our website in order to improve revenues and/or user experience.  While this implicit data capture is wonderful, you should be supplementing it with data that you collect directly from your website visitors.  Voice of Customer (VOC) is the term often used for this and it is simply asking your customers to tell you why your website is good or bad.  There are two main ways that I have seen people capture Voice of Customer:

1. Page-Based Comments – Provide a way for website visitors to comment on pages of your site.  This is traditionally used as a mechanism to get direct feedback about a page design, broken links or problems people are having with a specific page.  Unfortunately, most of this feedback will be negative so you need to have “thick skin” when analyzing this data!
2. Website Satisfaction – Provide a way for visitors to rate their overall satisfaction with your website experience (vs. specific pages).  This is normally done by presenting visitors with an exit survey where you ask standard questions that can tell you how your website is doing and compares your site against your peers.

There are numerous vendors in each of these spaces and the goal of this post is not to compare them, but rather discuss how you can integrate Voice of Customer data into your Omniture SiteCatalyst implementation.  In this post, I am going to focus on the first of the aforementioned items (Page-Based Comments) and specifically talk about one vendor (OpinionLab) that I happen to have the most direct experience with (their headquarters was a mile from my home!).  The same principles that I will discuss here can be applied to all Voice of Customer vendors so don’t get hung up on the specific vendor for the purposes of this post.

Why Integrate Voice of Customer into SiteCatalyst
So given that you can see Voice of Customer data from within your chosen VOC tool, why should you endeavor to integrate Voice of Customer and your web analytics solution?  I find that integrating the two has the following benefits:

1. You can more easily share Voice of Customer data with people without forcing them to learn [yet] another tool.  People are busy and you are lucky if they end up mastering SiteCatalyst, lest you make them learn how to use OpinionLab, Foresee Results, etc…
2. Many Voice of Customer tools charge by the user so if you can port their data into SiteCatalyst, you can expose it to an almost unlimited number of users.
3. You can use Omniture SiteCatalyst’s date and search filters to tailor what Voice of Customer each employee receives.
4. You can divide Voice of Customer metrics by other Website Traffic/Success Metrics to create new, interesting KPI’s.
5. You can use Omniture SiteCatalyst Alerts to monitor issues on your site.
6. You can use Omniture Discover to drill deep into Voice of Customer issues

I hope to demonstrate many of these benefits in the following sections.

How to Integrate Voice of Customer into SiteCatalyst
So how exactly do you integrate Voice of Customer data into SiteCatalyst.  For most VOC vendors, the easiest way to do this is by using Omniture Genesis.  These Genesis integrations are already pre-wired and make implementation a snap (though there are cases where you may want to do a custom integration or tweak the Genesis integration).  You can talk to your Omniture account manager or account exec to learn more about Genesis.

Regardless of how you decide to do the implementation, here is what I recommend that you implement:

1. Set three custom Success Events for Positive Page Ratings, Negative Page Ratings and Neutral Page Ratings.  These Success Events should be set on the “Thank You” page after the visitor has provided a rating.
2. Pass the free form text/comment that website visitors enter into an sProp or eVar.  If they do not leave a comment pass in something like “NO COMMENT” so you can make sure you are capturing all comments.  If you are going to capture the comments in an sProp, I recommend you use a Hierarchy variable since those have longer character lengths vs. normal sProps which can only capture 100 characters.
3. Pass the actual page rating (usually a number from 1 to 5) into an sProp.  I also recommend a SAINT Classification of this variable such that you classify 1 &2 as Negative, 3 as Neutral and 4 & 5 as Positive.  This classification should take less than 5 minutes to create…
4. Use the PreviousValue plug-in to pass the previous page name to an sProp.
5. Create a 2-item Traffic Data Correlation between the Previous Page (step #4) and Page Rating (step #3).  This allows you to see what page the user was on when they submitted each rating.

All in all, this is not too bad.  A few Success Events and a few custom variables and you are good to go.  The rest of this post will demonstrate some of the cool reports you can create after the above implementation steps are completed.

Share Ratings
As I mentioned previously, you [hopefully] have users that have become familiar with the SiteCatalyst interface.  This means that they have Dashboards already created to which you can add a few extra reportlets.  In this first example, let’s imagine that you want to graphically represent how your site is doing by day with respect to Positive, Negative and Neutral ratings.  To do this, all you have to do is open the Classification version of the Page Rating report (can be an sProp or eVar – your call) and switch to the trended view.  You should have only three valid values and I like to use a stack ranked graph type using the percentage to see how I am doing each day as shown here:

This graph allows me to get a quick sense of how my site is doing over time and can easily be added to any Dashboard.

You can also mix your newly created Voice of Customer Success Events with other SiteCatalyst metrics.  For example, while you could look at a graph/trend of Positive or Negative Comments by opening the respective Success Events, a better way to gauge success is to divide these new metrics by Visits to see if you are doing better or worse on a relative basis.  The following graph shows a Calculated Metric for Negative Comments per Visit so we can adjust for traffic spikes:

Find Problem Pages
Another benefit of the integration is that you can isolate ratings for specific pages.  The first way to do this is to see which pages your visitors tend to rate positively or negatively.  In the following report, you can open the Rating variable report (or Classification of it as shown below) and break it down by the Previous Page variable to see the pages that most often had negative ratings:

This will then result in a report that looks like this:

Alternatively, if you want to see the spread of ratings for a specific page, all you need to do is find that page in the Previous Page report and break it down by the Rating variable (or its Classification) as shown here:

Share Comments
As noted above, if you capture the actual comments that people leave in a variable, you will have a SiteCatalyst report that captures the first 256 characters of the comments visitors enter.  This report duplicates scheduled reports from your Voice of Customer vendor in that it allows you to share all of the comments people are leaving with your co-workers.  However, by doing this through SiteCatalyst, you gain some additional functionality that some VOC vendors don’t provide:

1. You can create a Traffic Data Correlation between the Comments variable and the Previous Page variable so you can breakdown comments for a specific page.  Therefore, if you have users that “own” specific pages on the website, you can schedule daily/weekly reports that contain comments only for those pages so they don’t have to waste time reading all of the comments left by visitors.
2. You can use the Search filter functionality of SiteCatalyst to scan through all of the visitor comments looking for specific keywords or phrases that your co-workers may be interested in.  In the example below, the user is looking for comments that mention the words “slow” or “latent” to be notified of cases where the visitor perceived a page load speed issue:

Set Alerts
Another cool thing you can do with this integration is set automated Alerts in SiteCatalyst so you can be notified when you see a spike in Negative Comments on your site.  This allows you to react quickly to broken links or other issues before they affect too many visitors (and help avoid #FAIL posts in Twitter!).  Here is an example of setting this up:

Review Problem Visits using Omniture Discover
Finally, if you have access to Omniture Discover, after you have implemented the items above, you can use Discover to do some amazing things.  First, you can use the unlimited breakdown functionality to zero in on any data attribute of a user that is complaining about your site.  For example, if you had visitors complaining about not being able to see videos on your site, you might want to see their version of Flash, Browser, OS, etc… or even isolate when the problem took place as shown here:

Additionally, you can use Discover to isolate specific comments and watch the exact visit that led to that comment.  This is done through a little-known feature of Discover called the “Virtual Focus Group.”  This feature allows you to review sessions on your site and see the exact pages people viewed and some general data about their visit (i.e. Browser, GeoLocation, etc…).  While not as comprehensive as tools like Clicktale, it is good enough for some basic analysis.  Here is how to do this:

1. Open Discover and find the comment you care about in the custom sProp or eVar report
2. Right-click on the row and create a Visit segment where that comment exists
3. Save the segment in a segment folder
4. Open the Virtual Focus Group (under Pathing in Discover)
5. Add your new segment to the report by dragging it to the segment area
6. Click “New Visit” in the Virtual Focus Group
7. Click on the “Play” button to watch the visit

Now you can watch how the user entered your site, what pages they went to and see exactly what they had done prior to hitting the Voice of Customer “Thank You” page.

Final Thoughts
So there you have it, a quick review of some cool things you can do if you want to integrate your chosen Voice of Customer tool and Omniture SiteCatalyst.  If you are interested in this topic, I have written a white paper with OpinionLab that goes into more depth about Voice of Customer Integration (click here to download it).  If you have done other cool things, please let me know…