Adam Greco's Blog at Web Analytics Demystified

I recently received an e-mail from a blog reader who was having issues tying their Orders in SiteCatalyst to Orders in their back-end system.  Here is a snippet from the e-mail:

I have a little issue in my own SiteCatalyst setup that I recently discovered.  Sad for me I had trusted the number of Orders for each day’s Conversion Funnel and recently I decided to validate the numbers in SiteCatalyst against what our back-end system has.  SiteCatalyst is 5%-10% understated each day which makes for a heck of a difference at the end of the month!  I’d rather be understated than overstated, but can you give me some ideas where I should look first?

Unfortunately this is an all too common problem I hear out there.  In this post I am going to share some ideas on how you can tackle this Order/Revenue validation issue head-on and make sure you can trust your critical Orders/Revenue data in SiteCatalyst.

Order ID eVar
If you have an online shopping cart, you should already be setting the s.purchaseID variable with a unique Order ID when an Order takes place on the website.  This variable is used by SiteCatalyst to ensure Order uniqueness.  Unfortunately, the downside of this variable is that it is not readily available in the SiteCatalyst interface.  It is available in DataWarehouse but not in regular SiteCatalyst reports or Discover.  Carmen Sutter (@c_sutter) has submitted  an idea in the Idea Exchange to change this, but until then, I recommend that you set what I call an Order ID eVar variable.  To do this, all you need to do is set the same value you pass to the PurchaseID variable to a custom eVar.  This will allow you to see all Orders and Revenue by Order ID from within SiteCatalyst and Discover as you would any other eVar.  Once you have done this, you can open up this new Offer ID eVar and add your Orders or Revenue Success Event as needed:

In the example above, we can see that most Orders have only one Order ID, which is what we want.  However, in this case, we can see that one ID was counted twice.  That may require some research and I like to schedule a report like the one above to be sent to me weekly so I can make sure nothing strange is going on.

Data Sources Setup
However, while adding an Order ID eVar is helpful in seeing if you are over counting Orders in SiteCatalyst, it won’t tell you if you are under counting Orders or  how close your SiteCatalyst data is to your back-end systems.  To do this, I recommend you use Data Sources.  As a quick refresher, Data Sources allows you to import external data/metrics into SiteCatalyst (see post link for more details).  In this case, I recommend that you import in a file from your back-end system into SiteCatalyst which contains your unique Order ID, the number of Orders (which should always be “1″) and the Revenue Amount.  When you import data via Data Sources, you include the date that you want the data to be associated with so it doesn’t matter if you import the data on a daily, weekly or monthly basis, but the more frequently you upload it, the better so you can find issues quickly.

Here are step-by-step instructions on how to do this:

• Create the Order ID eVar described above
• Create two new Incrementer Success Events and name them “Back-End Orders” (Type=Numeric) and “Back-End Revenue” (Type=Currency)
• Create a new Data Sources upload template (ClientCare or Omniture Consulting can assist with this).  You want to be sure to map the two new “Back-End” Success Events to the Data Sources template.  Even more critical, is that you want to include the newly created Order ID eVar in the Data Sources template.  If you do not do this, then you will not be able to see these two new Back-End metrics in the same Order ID eVar report that you have in SiteCatalyst (more on this later).

• When you are done, you should have a Data Sources template that looks something like this:

• Now all you have to do is work with your developers to have this file sent via FTP to the Data Sources FTP on a regular basis.

The Payoff
So by now, you are probably saying to yourself: “That’s a lot of work!”  No argument here!  However, hang with me as I share what the ultimate payoff is for doing this.  As you recall, our primary objective was to see if our online Order and Revenue data was matching what our back-end systems indicated.  Now that we have the Order ID eVar and two new “Back-End” Order and Revenue metrics, we have everything we need.  This is where the fun begins and we put it all together!

All you have to do now is to open the new Order ID eVar report and add all of the relevant metrics.  First, we will add the SiteCatalyst Orders and Revenue so we can see online Orders and Revenue by Order ID:

Next, we will add the two new “Back-End” metrics to the report and, since we were smart enough to include the Order ID eVar value in the Data Sources upload, SiteCatalyst knows which “Back-End” Order ID and dates line up with our online data:

Cool huh!  As far as SiteCatalyst is concerned, these offline metrics are connected to your Order ID eVar values just as if they had happened online.  Using this report, we can see if there are any differences between our online and offline data.  In the example above, it looks like the “Back-End” system had an order with $2,350 in revenue that wasn’t captured online.  Having this information makes it much easier to troubleshoot order submission issues.  You can even use DataWarehouse or Discover (only if you use Transaction ID Data Sources) to break down Order ID by browser, domain, IP address, etc… to see if you can figure out what is happening.  In addition, you can export this data to Excel and look at the totals to see how far off you are in general.

Finally, for the true SiteCatalyst geeks, you can create a Calculated Metric that divides Orders by Back-End Orders and/or Revenue by Back-End Revenue to see a trended % that each is off and set up Alerts to notify you if they deviate too much!  When you take into account this level of assurance all of a sudden the Data Sources work above might not seem like all that much in the long run!

Final Thoughts
If you sell products online, nothing is more critical than believing in your key metrics.  Even if you don’t sell online, the same principles here can be applied to lead generation forms, subscriptions or any other metrics you store in SiteCatalyst and also in your back-end systems.

Posted Monday, July 26th, 2010 | 10 responses | Share, Save or Email

[I apologize in advance for such a horrible blog post title, but I couldn't think of a succinct way to describe what I intend to cover.  Maybe one of you out there will have a better suggestion after reading the post!]

If your website is like many I have seen, you get a fair amount of daily visits and unique visitors, but it may be the case that a large number of your visitors don’t go beyond the first few pages of your site.  When I see this, I get very frustrated when I think about all I have done to get people to my site and optimized the site for my designated conversion goals.  But as web analysts, we need to put our emotions to the side and get down to the numbers.  Therefore, one of the things I like to do is to quantify how big of a problem my website has with visitors who only view a small number of pages.  In this post I will show you how to quantify this so you can begin to take action on addressing this issue.

The Setup
Before I get too deep into this topic, I’d like to setup the scenario since I think this will help it make more sense.  Let’s say that the main purpose of your website is to get visitors to view and complete lead generation forms.  Let’s also say that on your website you see that your most significant drop-off takes place after the third page of each visit.  In this situation, you might have lots of Visits and relatively few Form Completes so that your Conversion Funnel looks like this:

As you can see in this funnel, there is a pretty significant gap between Visits and Form Views.  While that presents a huge optimization opportunity, I like to break massive efforts like this into smaller chunks that I can work towards (or as Avinash points out – Micro-Conversions).  Since we noted earlier that a large portion of visits exit after three pages, wouldn’t it be nice if we could bridge the gap between our Visits metric and our Form Complete metric in the funnel above?  Having a middle ground between these Visits and Form Views might get our team to think about ways to turn more Visits into Visits of four pages or more which, depending upon your site, might be a step in the right direction.  In many sites I have worked with, there is a direct correlation between visitors viewing more pages and higher form conversion rates.

X+ Visits Explained
Now that we have set-up the situation,it becomes a bit easier to understand what I mean by “X+ Visits” since I am really saying that you can set a new Success Event metric which represents how many Visits your website gets where the visitor viewed more than “X” Pages.  What “X” represents is up to you and should be based upon your own data.  In this example, we will say that we are going to call it “4+ Page Visits” meaning the number of Visits in which Visitors viewed four or more pages.

The implementation of this is very easy for any good JavaScript developer since all that is involved is setting a Success Event as soon as each Visitor hits the fourth page of the session.  Once you have done this, you can update the conversion funnel shown above to look like this:

While this may not seem like much of a difference, here are some cool things you can do once you have this implemented:

• Create a Calculated Metric to divide 4+ Page Visits by Total Visits to see what % make it to four pages and trend this over time to see if you are getting better or worse

• Use the filter feature of the conversion funnel to see your funnel by Visit Number or Traffic Source (i.e. SEO) to see how each impacts the mix of Visits and Visits of four or more pages

• Create a calculated metric for the inverse (in this case three pages & fewer) by subtracting 4+ Page Visits from Visits.  I also like to pass both to Excel using the ExcelClient to create a stacked graph like this to show progress

Final Thoughts
There you have it.  If you find that you consistently have significant website drop-off after a few pages, hopefully, this new metric will help you better dissect what is happening so you can “Micro Conversion” your way to more Macro Conversion!

Posted Monday, July 12th, 2010 | One responses | Share, Save or Email

In my last post, I discussed how you can see how much money you are leaving on the table when it comes to the online shopping cart.  While I still have the shopping cart on my mind, I thought I would follow this up with a concept I call Product Pathing.  Product Pathing answers one of the questions I get from time to time:  How can I see the order in which website visitors are looking at my products or product categories?  The following will provide details on why you might want to do this and its implementation.

Why Product Pathing?
So why would you want to implement Product Pathing?  Here are a few reasons:

1. Understanding how visitors jump between products or product categories which helps you understand how visitors navigate your products
2. Seeing what products are viewed concurrently which helps you understand what cross-sell/up-sell opportunities might exist
3. If one of your website goals is to get visitors to view multiple products, you can measure how you are doing against that goal

There may be more reasons, but the preceding items should help you build a case for implementing this, especially since it is not difficult to do.

Implementing Product Pathing
So the standard way to see the answers to the questions above is to use page name-based Pathing reports.  You might find the page name of a particular product and then look at Pathing reports to see what visitors did after viewing the product.  However, I find that this approach does not work because there are so many pages on the website that it is impossible to sift through them all and isolate just product pages.  Therefore, I am going to propose the following alternative solution:

1. On all Product View Pages, in addition to setting a Product View Success Event and the Products Variable, pass the Product Name (or ID if that is all you have) to a new “Product” Traffic Variable (sProp). Be sure that you pass nothing but the Product Name (or ID) to this sProp.
   
2. After that is done, enable Pathing on this sProp

Believe it or not, that is all you have to do! By passing only the Product Name (or ID) to this new sProp, you will have a clean, new sProp that allows you to see Pathing reports on only Products like this:

Moreover, keep in mind that you have access to all Pathing reports so you get the bonus benefits of seeing the following as well:

• How often visitors looked at Product X and then didn’t look at any other Products (Exit % – 42.32% in this case)
• All paths containing Product X (Full Paths Report)
• What Products visitors see (if any) between Product X and Product Z (Pathfinder Report)
• How often did visitors see Product X and then Product Y (Fallout Report)
• Which Products were viewed first the most often (Entries) or last the most often (Exits)

A Few Other Cool Uses of Product Pathing
In addition to this, there are a few other cool things you can do:

• Instead of passing Product Names (of IDs), you can pass in Product Categories to see the same data at a higher level
• Instead of passing Product Name values at the Product View Success Event, you can set an additional sProp in which you pass Product Names when the Cart Add Event is set to see the order in which visitors add products to the shopping cart

Posted Tuesday, July 6th, 2010 | One responses | Share, Save or Email

(Estimated Time to Read this Post = 3.5 Minutes)

Imagine that you are in a retail store and you grab a bunch of items, bring them up to the counter and just as you are about to pay, you decide to push a few of the items off to the side and not include them as part of your purchase.  While this may not happen too often in real life, it happens quite often in eCommerce.  If you are a retail website, these discarded items can add up quickly!  In this post, I am going to show you how to quantify how much money you are leaving on the table.  For those not involved in a Retail site, I will also do my best to show how this concept can be applied to non-Retail sites.

The Standard Cart Process
So before we get to the more advanced stuff, let’s make sure we are all on the same page when it comes to the eCommerce shopping cart process.  Normally, here’s how it works:

1. Visitors view products on your website and you capture this with a Product View Success Event and store the products viewed in the Products Variable.
2. At some point, visitors add items to the shopping cart and you set the Cart Add Success Event and the Products Variable with the product ID or name(s).
3. Hopefully, visitors get to the Checkout Page and you set the Checkout Success Event and the Products Variable with the Product ID or name(s).
4. Finally, the order is completed and you set the Purchase Success Event which sets the Orders, Units and Revenue Success Events for each Product purchased.

Hopefully this is straightforward and if you sell online you have successfully implemented these steps on your site.  If so, you are ready to take things to the next level and do some stuff that is not traditionally done as part of standard eCommerce implementations.

How Much $$$ Left on the Table?
As the post name implies, in this scenario we would like to see how much $$$ we are losing online by website visitors leaving items in their Cart.  If you think back to the initial scenario above, this is equivalent to the Retail store adding up how much they could have made that day if no one had left stuff on the counter when they were checking out.  In addition to seeing how much $$$ is being missed out on, the store owner would probably want to know what products are being left to see if there are any patterns he/she could identify.  For example, it may be the case that items over $100 are left more often than products under $100, etc…

Well the good news, is that if you are doing business online, this much easier and you can see a lot more data on the items being abandoned and those who abandon them.  So here’s how you do it:

• When a website visitor adds one or more products to the shopping cart, in addition to setting the Cart Add Success Event (scAdd), you should set a currency Incrementer Event with the dollar amount associated with the items added.  As a refresher, an Incrementer Event allows you to pass in a numeric/currency value to a Success Event instead of using it as a counter.  By passing in the amount associated with the items added the Cart, you will have a new metric which represents the total potential that you could have made had no one left anything in the cart.  I call this new metric $$$ Added to Cart.



• Once this is done, you can compare this “$$$ Added to Cart” metric with your Revenue metric, either in a conversion funnel report or in a normal Conversion Variable (eVar) report by creating a Calculated Metric dividing the two metrics to see what % of $$$ Added to Cart turns into Revenue.

• If you want to be even more particular, you can set another incrementer event with the $$$ that the visitor has in the Cart at the time of Checkout.  However, if you find that you don’t have much loss between Cart Add and Checkout or between Checkout and Purchase, this may prove to be unnecessary.
• Finally, since you are setting the Products variable with the Cart Add event already, when you compare these two metrics, you can easily break it down by Product (or any other eVar variables you have set previously).

Beyond Retail
As promised, I wanted to touch upon a few ways you could use this same concept if you manage a non-Retail website.  Here are a few that come to mind:

1. On a Financial Services site, pass in the total loan amount a person is requesting and compare that to how much they are eventually loaned.
2. On a Media site, pass in the total amount of advertising your site could have earned if all ads were clicked.
3. On an Auto site, pass in the total value of cars visitors configure to see your max potential.
4. On a Lead Generation site, pass in a value for ever visitor who starts completing a lead form.
5. On a Travel site, pass in the total value of trips planned online and compare it to the amount actually booked.
6. On a Manufacturing site, pass in the total Bill of Materials value the visitor has added.

As you can see, the concept of seeing what your high-end potential is and comparing it to actual performance can be applied to almost any website and gives you another data point for comparison.  I like using this metric better than Visits or Unique Visitors since it is not realistic that you are going to convert every person who comes to your site.  However, once a visitor takes some more deliberate actions, they are self-qualifying themselves, and therefore, capturing their potential revenue streams gives you a high, but realistic goal to strive for and a KPI that you can use to see how you are doing over time.

Final Thoughts
So there you have it.  Just a quick, easy way to add some more data to your all-important shopping cart process.  In general, I feel like Incrementer success events are under-utilized by SiteCatalyst users so hopefully this example helps to get your mind working in new and inventive ways to use them…

Posted Monday, June 28th, 2010 | 2 responses | Share, Save or Email

(Estimated Time to Read this Post = 3.5 Minutes)

From time to time, I will see people talking about some of the limitations of SiteCatalyst Traffic Data Correlations on Twitter and blogs.  Below is the most common request I see:

For those of you not familiar with Correlations, they are a way to break one Traffic Variable (sProp) down by another sProp, assuming that both are set on the same page (image request).  Unfortunately, in SiteCatalyst, the only metric you can see for Correlations are Page Views.  In the example below, I can use a Correlation to see what page a visitor was on when searching for a specific phrase in the internal search box:

While this is handy, what if I wanted to see how many Visits people searched for this phrase from each page or better yet, how many Daily, Weekly, Monthly Unique Visitors did this?  Currently, the only way to get at this information is to use DataWarehouse or Omniture Discover.  However, the following section will show you a handy little hack to get this information directly from SiteCatalyst…

The Hack
So in this scenario, we will say that we want to see how many Weekly Unique Visitors searched for the phrase above from each page on our website (as in the example above).  Here is the trick to doing this:

1. Just as in a Correlation, you must have both data points you want to correlate available on the same page.  In this example, it is Previous Page Name (from the GetPreviousValue JavaScript Plug-in) and the Internal Search phrase
2. Once you have the two data points available, create a new Traffic Variable (sProp) and concatenate the two values using a separator.  In this example, if the user searched for the above phrase from the Japan Customers page, the value would be “キーワード検索:SFDC:jp:customers”
3. After you have passed the concatenated value to the sProp, contact your Omniture Account Manager and tell him/her that you would like to enable Visits, Daily Unique Visitors, Weekly Unique Visitors, etc… on that sProp

When all that is done, using the example above, you will have a report that looks like this:

The confusing part of this hack is that you won’t actually use a Correlation report anymore.  You will no longer open one report and break it down by another report, but instead you will simply open the new sProp report and add the metrics you want to see.  In the report above, I have added Page Views, Visits and Weekly Unique Visitors and searched on the specific Internal Search phrase for which I am interested.

However, as you can imagine, you could have a lot of unique values in this report.  One caveat is that you need to make sure you don’t exceed the SiteCatalyst variable limit which is 500,000 unique values per month.  In this example, you would want to make sure that the combination of Page Name and Search Term does not exceed 500,000 per month, but for most sites this shouldn’t be a problem.

Another thing to keep in mind is that the above scenario is just one example.  This “hack” is by no means limited to Internal Search Terms and Pages.  Here are some other examples of what you can do with this “hack:”

• Unique Visitors/Visits who saw a specific page by visit number (i.e. Home Page:Visit 1, Home Page: Visit 2, Demo Page: Visit 1, etc…)
• Unique Visitors/Visits who searched for a term by visit number (i.e. Pricing: Visit 1, Pricing: Visit 2, Demo: Visit1, etc…)
• Unique Visitors/Visits who saw a specific page by country (i.e. Home Page:US, Home Page:UK, Demo Page:US, etc…)
• Unique Visitors/Visits viewing a specific product page by search engine (i.e. Product X:Google, Product Y: Google, Product X:Yahoo, etc…)
• Unique Visitors/Visits viewing a specific product page by search keyword (i.e. Product X:walmart, Product Y:walmart, Product X: walmart.com, etc…)

These are just a few examples and my advice is to look at whatever you are currently correlating (in Admin Console) and determine the items for which you would like to see Visits and Unique Visitors.

Advanced Users
For advanced users out there, I wanted to call out a few more things you can do with this concept:

• If you don’t have a lot of unique combinations, you can add multiple correlations to the same sProp and use the search box to find the item combinations you need.  For example, you may use the Internal Search & Page example shown above, but also store Page Name & Visit Number combinations in the same sProp.  As long as all of your data is underneath the 500,000 unique value monthly limit you are ok.  Alternatively, you can multiple sProps assuming you have enough variables that can have Visits and Unique Visitors enabled remaining in your contract.
• With this alternative approach, you can also view Trending Reports for each of your combinations if you enable Pathing.  This means that you can trend Visits or Unique Visitors for any combination (i.e. Weekly Unique Visitors who view Product Page X on Visit Number 1 over the last 90 days).  This temporarily solves the following Idea Exchange item (Allow Trended Versions of Correlation Reports)

Final Thoughts
So there you have it.  Just a quick “hack” that allows you to get a bit more information out of SiteCatalyst.  In the future, perhaps Omniture will allow you to see Visits and Unique Visitors right form the normal Correlation interface (please vote for this here: Provide Visits and Unique Visitors in Correlation Reports), but in the meantime, hopefully this will help bridge the gap…

Posted Monday, June 21st, 2010 | 2 responses | Share, Save or Email

In the past, I have written about Bounce Rates, Traffic Source Bounce Rates and Segment Bounce Rates.  Hopefully this will be my last post related to Bounce Rates, but I recently found a “hack” to calculate and trend a Site Wide Bounce Rate in SiteCatalyst so I thought I would share it.  I define Site Wide Bounce Rate as the total number of Single Access Visits divided by the total number of website Visits.  Unfortunately, for some reason, this metric is very difficult to wrestle down in Omniture SiteCatalyst because you cannot view Pathing metrics (i.e. Entries, Single Access) in Calculated Metrics unless you are within an Traffic (sProp) report that has Pathing enabled.

To date, the way I have reported on Site Wide Bounce Rate was by pulling Visits and Single Access data into Excel using the SiteCatalyst ExcelClient.  Once there, I could divide the two and if I wanted to see it by day (or week or month), all I needed to do was to pull both metrics by day.  It was straightforward, but I could not add this to my SiteCatalyst Dashboards.

The Hack
So let’s say that you want to report a daily/weekly/monthly trend of your Site Wide Bounce Rate and add it to one of your executive dashboards.  Here are the steps:

• First you need to create the required calculated metric.  In this case you want to divide Total Single Access by Total Visits (or Total Entries which is the same thing).  I would recommend making this a Global Metric so all of your users have access to it going forward:

• Once this metric is created, open your Pages report, click the Add Metrics link and add the new Site Wide Bounce Rate metric to your list of metrics.  It will be under the Calculated Metrics area.  Place this new Site Wide Bounce Rate metric so it is the first metric and then add your regular Bounce Rate metric and finally add the Page Views metric and click the small triangle to sort by Page Views.  When you are done, it should look like this:

• When you click OK, you will be able to see a report that shows your most popular pages, the Bounce Rate for each page and the overall Site Wide Bounce Rate.  This report is handy for seeing how each page is doing in relation to the Site Wide Bounce Rate.

• However, our original objective was to see the trend of the Site Wide Bounce Rate and add it to a dashboard, so let;s get back on track.  To do this, all you have to do is click the “Trended” link shown in the report above.  As is always the case, trending will show you the left-most metric trended over your chosen date range (which is why it was important to put Site Wide Bounce Rate in the first metric slot!).  After clicking it, you will see a report that looks like this:

So the resulting graph is your Site Wide Bounce Rate and you can now add this to any SiteCatalyst Dashboard.  However, as you recall, I mentioned this is a “hack” so if you look closely you will see a bunch of pages in the data table for this report.  What is strange is that the values for each row are the exact same.  This is the place where you can see how much of a hack this is.  This data is pretty much useless so I recommend just adding the graph to your dashboards and ignoring the data table.  Perhaps in the future Omniture will let us add this type of Calculated Metric to the “My Calc Metrics” area so we don’t have to take such a convoluted path to add this trend graph to a dashboard!

Final Thoughts
So there you have it.  A quick hack in case you ever need to calculate Site Wide Bounce Rate for your HIPPO’s!  Enjoy!

Posted Monday, June 14th, 2010 | 13 responses | Share, Save or Email

(Estimated Time to Read this Post = 4 Minutes)

If you are in the Web Analytics space, besides tracking what people do on your website, hopefully you are actively doing testing and content targeting to try and improve your conversion.  If you are an Omniture customer, you might be using their Test&Target product or you may be using Google’s Website Optimizer.  If you are just getting into the testing area, you may simply be using an eVar to see how your tests are performing.  Regardless of what tool you are using, there is a common question that arises in the testing/targeting area.  Here is the scenario:

1. You come up with a great hypothesis you want to test
2. You run a test and see awesome results (say a 10% uplift in conversion)
3. You broadcast it to your company only to hear the inevitable “well that was just a test…how do you know we’ll see the same result in real life?”

As a web analyst, this can be infuriating and can be compounded by the fact that you often cannot simply run with the winning recipe and show the results in your testing tool because:

• You may be running multiple tests and things can get confusing
• You may want to apply what you have learned from the test to many places on your website which may or may not have the required “MBoxes”

In reality, it may take time for you to take your awesome test and let it out “into the wild” and when you do so, how can you prove that the uplift you saw in your test will actually occur over the next year on the website?  The following will tell you exactly how you can do this and hopefully put the naysayers in their place!

How To Prove Your Test Results
So now that I have framed the situation, let’s learn how to do it.  Our objective is to prove the long-term results of a test we did using our chosen testing/targeting tool.  In this example, let’s imagine that your website has twenty forms on it and you have just done a test showing that if you reduce the number of fields on a form, you can see a 15% uplift in Form Completion Rates.  This test was conducted using Test&Target for three weeks with a high level of statistical confidence (+95%).  Now you want to go ahead and take five of the twenty forms and remove the same fields you did in the test for the next three months and see what happens.  One way to do this would be to add lots of “MBoxes” and use Test&Target to deploy the winner in hopes of seeing the same lift results, but in this example, let’s assume that your conversion team has closed the books on this test, moved onto other tests and has told you that you now need to work with the web team to reduce the fields on your five forms.

So what do you do?  How will you know if these five forms will really see a 15% uplift over the next three months?  All you need to do is the following:

1. Create a new Testing eVar (not the T&T eVar)
2. On each of the five forms you modify on your website, pass in the name of the the test that it was based on to this new eVar.  This may be the name of the winning T&T recipe or you can use any descriptive name you’d like.  In this case, we’ll pass in the value “Remove Form Fields Test”
3. Set the eVar to “Most Recent Value” and expire “Never” in the Admin Console

That’s it.  Now when you open this new Testing eVar report, you can see how these five new forms are doing with respect to Form Completion Rate (assuming you have the right Success Events set – in this case Form Views and Form Completes).  When you look in this new eVar report, all forms that were not modified based upon a testing initiative will fall into the “None” row so you can easily compare those forms that are based upon testing with those that are not:

In the preceding example, we can see that the “Remove Form Fields Test” seems to have about a 17% uplift in Form Completion Rate after it was fully deployed so we are doing even better than the 15% expected!  What’s better, is that if you repeat this process every time you make changes to things on your website based upon testing, you can see how each is doing:

And, if you look at them all together, you can show your boss at the end of the year how much uplift you have been responsible for overall!  In this example, if we look at all of the tests we have implemented, we are seeing a cumulative uplift of 16.2% over forms that are not based upon any testing.  This is a great way to show the value of your conversion efforts and justify more headcount, get promoted, get more budget, etc…  In fact, you can show your boss, that if all of the “Form Views” on your website were, in this case, seeing optimized forms, you could produce 5,800 Form Completes instead of the 5,000 you are currently getting at the lower Form Completion Rate.

The only downside of this solution is that it might actually show you that something you expected to have an uplift, in reality didn’t.  For example, in the preceding screen shot, the “Form Headline Bold” change doesn’t seem to be pulling its weight (losing against the control)  and may need to be revisited.  However, even though this is disappointing, it is great information to have since it might prompt you to do some further testing in Test&Target and abandon the losers.

Finally, if you want to get a little more advanced, you could also apply SAINT Classifications to this new Testing eVar and group your tests into types (i.e. “Field-Related Tests” or “Color Related Tests”) so you can calculate the uplift of each type and see which ones you may want to focus on going forward.

Final Thoughts
So there you have it.  As a rule of thumb, I would build a step for passing in the Test Name a change was based upon into a Testing eVar into your conversion testing process so that you can look at how your tests ultimately perform.  While this will add one small step to your overall process, I think that in the long run you will be happy that you have this variable to show how your team is doing…

Posted Monday, June 7th, 2010 | 5 responses | Share, Save or Email

(Estimated Time to Read this Post = 2.5 Minutes)

From time to time, I hear people talking about Data Extracts in SiteCatalyst (especially on Twitter).  In this post, I thought I’d review them in case you are unsure of what they are and how they are used.

What Are Data Extracts?
The good news is that if you know how to use the SiteCatalyst ExcelClient then you are essentially an expert in Data Extracts, you just may not know it!  Behind the scenes, Data Extracts are really just a different way to access the engine that powers the ExcelClient data blocks with which you are already familiar.  The main difference between the ExcelClient and Data Extracts is that Extracts are normally used to have a report e-mailed on a regular basis without having to go into Excel (especially good for Mac users!).  Any report for which you can create a data block in the ExcelClient can also be accessed using the Data Extract icon in the SiteCatalyst toolbar:

If you cannot see the icon shown above in a SiteCatalyst report that means you cannot create a Data Extract for that report.  Once you click the icon, you will see a new screen that looks like this:

If you have used the ExcelClient, this should look familiar and you use this screen to choose how much data and what metrics you’d like to see.  Whatever settings you choose on this screen are what you will be stuck with (except for date which I believe is floating).  Finally, in the example below, notice that Data Extracts (and the ExcelClient for that matter!) have access to the same Classifications, Correlations and Subrelations that are available when using the normal SiteCatalyst user interface (click the green magnifying glass icon).  In this example, I am showing a correlation between Day of Week and Hour of Day:

Once you have configured your Data Extract, the last screen will ask you if you want to have it send via e-mail, added as a bookmark or both.  In a minute I will share with you why it is advantageous to store Data Extracts as bookmarks, but I recommend keeping any Data Extracts you create in an “Extracts” folder in case you ever need them again.

That’s it.  You’re done.  If you have e-mailed yourself the report, it will arrive shortly thereafter.  However, I have to explain something that often perplexes folks.  If you chose to add a bookmark, inevitably at some point you will open that bookmark using your bookmark toolbar and be facing a screen that looks like this:

If you are like me the first time I saw this, you might be a bit confused.  I expected to see the actual report, but that will never happen.  This screen is only to be used to modify your Data Extract and to open it so you can re-send it to yourself or others.  At first, I decided that this devalued my decision to store my Data Extract as a bookmark, but don’t despair because SiteCatalyst makes up for this by allowing you do something really cool with bookmarked Data Extracts.  If you open the ExcelClient and choose to insert a new Data Block, you can create a new one or you can choose from any Data Extracts you have created (or shared ones) as shown here:

Notice that the new Data Extract we just created appears here.  All we have to do is to click the “Insert” link and it will add the data block to our Excel spreadsheet and run the query:

This saves you having to re-create it in the ExcelClient and you can still edit it to suit your needs after it has been inserted.  The only bummer is that you cannot tie the Data Extract data block to cells in Excel so if you are ever looking for dynamic data blocks, start them in Excel, not as Data Extracts.

So that is pretty much all you need to know about Data Extracts.  One final note – it is my understanding that Data Extracts do not work with the new Omniture ReportBuilder, but perhaps something similar will be rolled out in the future.

Posted Tuesday, June 1st, 2010 | No responses | Share, Save or Email

(Estimated Time to Read this Post = 2.5 Minutes)

In my last few posts I have been delving into Web Analytics & CRM (Customer Relationship Management) integration.  In my first post I described how you can pass Web Analytics Data to your CRM system to help your sales people.  In my last post, I described how you could pass CRM data like Leads, Opportunities and Revenue into your Web Analytics tool.  In this post, I will round out the trilogy by describing how you can use CRM data as Web Analytics meta-data to enhance your Web Analytics reporting.

My Golf Handicap Story
Since most people don’t often like talking about meta-data, I will begin by sharing an easier to understand story which first taught me how interesting integrating CRM and Web Analytics data could be.  Back when I managed the website for the CME, we had situation in which we were trying to sell tickets for a major golf tournament.  Unfortunately, the event was nearing and we still had lots of tickets to sell.  At the time, I recalled that, for registered website users, we had golf handicap as one of our CRM fields in our Salesforce.com system (our customers were traders and spent a lot of time golfing!).  I had recently worked on capturing each customer’s website ID in SiteCatalyst and also placing it in our CRM system.  Suddenly, the light bulb went on in my head…why not upload golf handicap as a SAINT Classification of the website ID I had in an sProps and eVar in SiteCatalyst?  I created a SAINT Classification table that passed in the raw handicap and also grouped it into buckets like this:

Whereas previously I could see what pages each website ID had viewed on the website, I could now expand that to see the same data for this new golf handicap Classification of that variable.  The result was a report like this, in which I could see the most popular pages for website visitors by golf handicap:

From there, all that was left to do was to target some ads on those pages and voilà, the tickets were soon gone!

For me, this was more experimental than anything else, but it was the catalyst (no pun intended!) which helped me see the power of  integrating CRM and Web Analytics.  Of course back then there were no API’s to help pass data between systems, but nowadays, this is much easier (i.e. Genesis integrations).  With this in mind, let’s take a look at a few more examples of how you can take advantage of this concept.

Examples of Passing CRM Meta-Data to Web Analytics
Now that you get the general idea, I’ll walk you through some other examples of enriching your Web Analytics data by bringing in CRM meta-data.  Let’s assume that you have done the steps outlined in my last post and have made a connection between your Web Analytics visitors and your known CRM prospects/customers.  Using the primary key described in my last post, you can export whatever CRM fields you care about from your CRM system and import them into your SiteCatalyst implementation as SAINT Classifications.  Here, you can see that I have decided to export Industry, # of Employees, Lifetime Value and a Lifetime Value grouping (to make my reports more readable) from my CRM system and import them using the following SAINT file:

Now that I have done this, I can open my Lead Gen ID report in SiteCatalyst and look at any of these CRM fields as Classifications.  Here is a view of some of my Success Events by Industry:

Here is the same data viewed by # of Employees:

Here is the same data viewed by Lifetime Value:

The same concept can apply if you are using other Web Analytics tools.  Here is an example of viewing reports in Google Analytics by Job Title (in this case filtering for CIO’s):

Final Thoughts
As you can see, once you have made the connection between your Web Analytics and CRM system, there are lots of creative things you can do with respect to augmenting your traditional web analyses.  I know a lot of people also do this in tools like Quantivo or Omniture Insight, but I hope this was helpful to see some of the ways to do this if you only have access to SiteCatalyst.

Posted Monday, May 24th, 2010 | No responses | Share, Save or Email

(Estimated Time to Read this Post = 5 Minutes)

In my last post, I explained a bit about CRM and how you could improve CRM by passing Web Analytics data into your CRM system.  In this post, I am going to cover the reverse angle  – passing CRM data into Web Analytics.  Since most of you reading this are web analysts, I think you will find this post more relevant, but I think it is important to understand both sides.

Why Pass CRM Data into Web Analytics?
As I mentioned in my last post, we web analysts get lots of great information about website visitors, but for many companies (especially B2B), the richest data resides in the CRM (Customer Relationship Management) system.  If you want to be relevant in your organization, it is always best to be as close as possible to the $$$ and that often means playing nicely with CRM systems.  Don’t get me wrong,  showing your CMO that you can lift form completion rates by 200% through optimization is awesome, but if you can show him the revenue impact of it right there in your Web Analytics tool, you will be a rock star!  Additionally, I will show that if you don’t have actual revenue-generating events on your site (eCommerce and Media sites have this easy!), then not doing this could actually result in Web Analytics data causing incorrect business decisions…

Passing Post-Website Data from CRM to Web Analytics
OK.  So there are many different ways to merge CRM and Web Analytics data including passing data from both into a massive Marketing data warehouse (or Omniture Insight), but just for the purposes of this post, I am going to assume that you are a SiteCatalyst person and want to get something done relatively quickly.  In this scenario, we’ll assume the following:

• You want to see which of your website visitors completing lead forms on the site evolve into Leads, Opportunities and Revenue
• Your CMO has charged you with capturing all of the different marketing channels and asked for your opinion on where the company should invest to get the most Revenue
• You are tracking the various sources of traffic you receive and using SAINT Classifications to roll each up into a high-level marketing channel (SEO, SEM, E-mail, etc…)

Given all of this, you might have a SiteCatalyst report that looks like this:

As a web analyst, at this point, it looks like we might want to invest more in our E-mail program since that seems to be converting the best.  Without CRM integration, that would probably be as far as we could go.  But let’s now dig a little deeper.  As I mentioned in the last post, when website visitors complete a form, we have a brief moment in time when we can connect our website data with our CRM data.  Most CRM tools allow you to capture leads and set a unique ID for each form completion.  At the same time, Omniture SiteCatalyst has a really cool feature (that many don’t use enough!) called Transaction ID.  I highly recommend you read my full post on Transaction ID, but at a high level, it allows you to set an ID to a special SiteCatalyst variable and then days or weeks later, upload [normally offline] metrics into SiteCatalyst.  The magic of Transaction ID is that when you upload these metrics later, they are tied to the eVar values (sorry – no sProps or Participation) that were present at the time the Transaction ID was set.  That means that if a website visitor had a City eVar value of Chicago, a Traffic Source eVar value of Paid Search and a Visit Number eVar value of 3, then any offline metrics you import will also be tied to Chicago, Paid Search and Visit Number 3 in the respective eVar reports.  This means that if you set the CRM ID associated with a website form completion, you now have a primary key (think Rosetta Stone!) that can connect your Web Analytics data to your CRM data!

So what does this mean to you?  Following our preceding example, let’s assume that you have made this connection and later imported all of the new leads your CRM system has seen along with the status (i.e. Qualified)  of each into SiteCatalyst (these new metrics would be Incrementor Events).  This gives you a new metric named “Qualified Leads” that you can now see in SiteCatalyst reports and since you used Transaction ID, these imported CRM metrics are correctly attributed to all eVar reports in your implementation.  The result is that you can now open a report similar to the one we saw above, but now it has “Qualified Leads” instead of Form Completes and a new Calculated Metric that divides these Qualified Leads by Visits:

The icons above the report show where each data point comes from and as you can see, the last column is truly magical in that it is combining data from two disparate systems (Cool huh?)!  Once we have this, we can see that even though E-mail looked to be the best channel a few minutes ago, it now appears that SEM is where we want to spend our money.  It turns out that E-mail generates form completions at the highest rate, but perhaps those form completions are all junk!

However, I like to go as far downstream as possible and nothing is better than cold, hard cash!  Applying the same principles, we can import Qualified Opportunities, Potential Pipeline, but the CRM metric that trumps them all is Revenue.  By uploading Revenue via Transaction ID, we can see how much $$ we got from each Lead Form completed on the website and tie it to any eVar value we have – in this case marketing channel/traffic source.  The following report shows the result of this:

Again, we see that some data is coming from SiteCatalyst and some is coming from our CRM system.  Our new Revenue/Visit Calculated Metric can be used to see that, in the end, it is really SEO that provides the most Revenue/Visit and maybe we should consider additional investment there.  Please keep in mind that these examples are simply meant to illustrate the concept and show the value in adding CRM metrics to your Web Analytics tool.  Finally, don’t forget that Transaction ID data is available in Omniture Discover so you can slice and dice this data even further there!

Targeting Based Upon CRM Data
Another really cool integration between CRM and Web Analytics is in the area of Test&Target.  For those not familiar with Test&Target, it is an Omniture tool that lets you test and dynamically target content to website visitors based upon what you know about them.  It is commonly used to optimize your website success metrics.  However, this can be extended by importing in CRM data so that your targeting is based upon both online and offline data.

Let’s walk through an example.  Imagine that a website visitor named Bill has been to your website a few times, looked at a few of your products and completed a lead form.  Next, Bill spoke to your sales representative and is at “Stage 3″ of the sales process (the discovery phase).  Over the next few weeks, meetings take place and Bill comes to the website occasionally (your sales team would know when and exactly what he is doing if you read my last post!).  But now let’s say that Bill is in sales “Stage 9″ which is the final stage before the deal is won or lost.  We know what products he wants, we know he is close to making a decision, we know how big is company is, etc…  If we knew all of this, what would we want to show him the next time he arrives at our website?  Here are a few things I would show to Bill on my home page when he (and only he) arrives on it:

1. Case studies related to his industry
2. ROI calculator for the product Bill is interested in
3. Links to community content to show Bill that he would be well taken care of if he were to be a customer
4. A time-sensitive offer (“Buy in the next 24 hours and get XX% off”) – You could even address him as “Bill” but that might freak him out!
5. etc…

The point is that if you can get the rich customer data related to Bill and multiply this to all of your prospects, each one could see more personalized content that helps move them further down the sales funnel.  You can even track how often they see these “recipes” and track the success of your intelligent targeting.  If you are interested in this type of CRM-based targeting I suggest that you contact @brianthawkins who is a Test&Target Jedi-master…

Final Thoughts
Hopefully this sparks some ideas about ways in which you can enrich your Web Analytics data by adding CRM data to the mix.  In the next post I will cover ways in which you can import CRM meta-data into your Web Analytics tool to augment your current web analyses.

Posted Monday, May 17th, 2010 | 6 responses | Share, Save or Email