How important is the latency reporting problem for PPC discussed earlier this week?

Take a look at this report snippet from onlynaturalpet.com. As part of our initial work on the account we’ve been cleaning up the organization of many ad-groups, so since early December there has been quite a lot of campaign and ad-group reorganization going on.

Weeks later reviewing a portion of the January results, we see clearly the impact of the way latent orders are handled – a bunch of ad-groups with zero spend, zero clicks, but orders and revenues.

Of course this only shows half the story. Each of those ad-groups has been recreated in a different location within the campaign, so there is a corresponding ad-group elsewhere in the report which shows both clicks and revenue for this same time frame, but as compared to peer ad-groups that weren’t part of the clean-up, the new ones suffer from much lower revenue (because all their trailing revenue is still being reported here in the old groups).

For beyond the fact that reporting both clicks and revenues when they happen as opposed to attributing revenue to the click that provoked it distorts reports by assuming that costs and revenues are constant when they’re not (see the earlier post for a better explanation), we have the problem that after making organizational changes your reports have severe limitations (they’re sort of worthless) until many weeks after the changes.

This post is part of a case-study series on the Commerce360 management of paid search campaigns for onlynaturalpets.com. It is being done with the kind permission of Only Natural Pet Store, and some data has been changed to keep PetSmart guessing. For your convenience, we’re keeping a list of all posts in the series.

Another interesting fact learned at the Channel Intelligence Retail Summit: According to David Stewart, Google’s Platform Marketing Manager, Commerce & Analytics – text ads with the Google Checkout icon are seeing a 10% higher click-through rate and a 40% higher conversion rate.

That’s a very nice additional CTR and obviously a huge average improvement in conversion rate.

Since higher CTR drives ‘quality score’ and should raise average position, or allow you to maintain a position at a lower bid, Google is delivering quite an incentive to put at least some of your payment business through G-Checkout. I’m going to see if our case-study client at OnlyNaturalPet.com would be willing to add Google-Checkout support so we can see the impact for ourselves.

While they clearly want to drive those transactions, this does seem a bit unfair to advertisers spending hundreds of thousands or millions to run paid ads but who don’t want to adopt Google Checkout, and now find themselves effectively penalized as a result (for those keywords where competitors have the Checkout icon).

If you look at a page of results with the Google-Checkout enhanced listings it’s clear they draw extra attention. Once there are a bunch of them on the page I think they actually make it harder to read the text ads. I wonder whether these icons will last for enternity or only until the Google Checkout program reaches some target adoption level.

Even more interesting, what if Google let you ‘enhance’ your text-ad with a little graphic icon, either from a library or of your own choosing, for some small incremental fee? Sort of like eBay does with listing enhancements. Certainly would clutter and de-beautify the page, but if they’re willing to do it for Checkout, how about giving us all a shot?

It’s been a while since I’ve spent time in Omniture Discover 2, but today I had a question that I thought it could answer.

OnlyNaturalPet.com is very interested in the number of new customers being driven via paid search. So I popped into D2 and quickly segmented units and sales for paid search by all visitors vs first time visitors.

Sure enough, we’re getting a pretty high count and percentage of new website visitors via our PPC programs. I can’t find first time buyers/customers yet, but I think that data may be here somewhere.

I was also reminded of the great drill-down benefit of D2 as I easily made my way through the layers of data.

As the screen shot above shows (click to enlarge) I can clearly display an entire path: Engine-Campaign-AdGroup-Keyword-Query (as covered in several recent blog posts) and see the actual products purchased for each query. That is amazing detail.

I should also point out that moving up-down-left-right within Discover is very fast. I hadn’t though about D2 last week when writing the post on how tough and slow navigation was for PPC. I’m not yet sure I can get to all the metrics I’d need for daily use here (more on that in a later post), but D2 sure allows fast and fluid omni-directional navigation.

Next, I’m going to rebuild my most favoritests calculated metrics in D2, and find/refind everything else this thing can do.

This post is part of a case-study series on the Commerce360 management of paid search campaigns for onlynaturalpets.com. It is being done with the kind permission of Only Natural Pet Store, and some data has been changed to keep PetSmart guessing. For your convenience, we’re keeping a list of all posts in the series.

Here’s what vision looks like:

“I like the ITouch enough that I just sent an email to the Mavs IT head to see how we fans with ITouchs could leverage WiFi in the American Airlines Center before , during and after Mavs games, HDNet Fights and other events…”

Mark Cuban, owner of the Mavs, said that on his blog today. Many other business owners would be wise to follow his example and get ahead of the impact/possibility of that browser format.

I was talking to someone today and suggested that the impact of iPhone on ecommerce was vastly under estimated in terms of what was going to happen in the coming 12 months. Smart websites are going to start crafting iPhone formatted web pages and they’ll be used in droves.

Reading Cuban’s remarks made me think that retailers should have in-house WiFi with detailed product information in iPhone/iTouch format, the kind that once-upon-a-time you could get from available and knowledgable sales people.

Put a simple code on each product, go to iphone.storename.com/productid and get all your questions answered. Wouldn’t that be great?

The fact that many paid search visitors come to the website multiple times from multiple sources is well known, but solid stats about this aspect of user behavior are hard to come by. I’m down at the Channel Intelligence Retail Summit in Orlando, and they’ve just shared some interesting related data.

The following numbers are based on data CI collects from hundreds of manufacturers, retailers, and shopping comparison engines. From within that world, these are the number of people who used different shopping channels/sources before their purchase.

1. One Channel Before Purchase – 37%
2. Two Channels Before Purchase – 25%
3. Three Channels Before Purchase – 16%
4. Four Channels Before Purchase – 8%
5. Five or More Channels Before Purchase – 14%

Two things to note:

• Each visit through an email, organic or paid search, shopping comparison engine, or retailer site counts as a channel visit. In other words, if you came through two different shopping engines those are counted as two different channels in the above list even though they’re the same channel.
• This data does not document the number of visits to a single website. A user who went through four channels to four different sites before purchasing is in the ‘Four Channels’ data.

This is really interesting data, and generates a lot of thoughts on the issue of attribution. More thoughts on that in a future post.

Many of the visitors who click your paid search ads don’t complete a purchase on the day of their click and initial visit, but instead come back some number of days later and buy.

The paid search engine tracking tools and major web analytics packages can (generally) track these delayed purchases. They assign credit for this delayed revenue back to the paid search channel and even the engine, campaign, ad-group, and keyword responsible for the click.

Most use a 30-day timeframe during which delayed purchases are credited properly. Some allow you to specify the timeframe – SiteCatalyst/SearchCenter for example allows you to track delayed purchases for up to one year. (Set the evar expire date in the Admin control panel.)

But there is a timing problem in PPC reports. They show the clicks/expenses for the requested period, and the orders/revenue that took place in the requested period. But there is no requirement that the reported clicks cause the reported revenue. In fact, it’s an open secret that in many cases they don’t.

(There are other issues causing inaccuracies in these reports too, as posted earlier.)

There are three sources for the data in a paid search report:

• Left-Over Transactions. Revenue from clicks/visits that occurred prior to the initial reporting date. The clicks and expense of these clicks is not included on the report.

---

• In-Period Transactions. Clicks and their resulting revenue when both occurred inside the reporting dates.

---

• Incomplete Transactions. Clicks and their expense for which some of the orders and revenue occur after the final reporting date. These orders and revenue are not included in the report.

So for a report on paid search ‘last month’ in a company which has trailing sales for about three weeks after the initial click (not at all atypical), the report would include:

• A stream of left-over transaction revenue, starting with 21-days worth the first day of the month stretching all the way into the third week of the month.

---

• A good number of in-period transactions where the click occurred on or after the 1^st of the month and the sale closed by the last day of the month

---

• The full expenses but an unknown amount of revenue for incomplete transactions from every day except the first week of the month. It should be noted that the revenue from these incomplete transactions never ‘fills-in’ for the ‘last month report’. Even if you wait the full 21 days and then run it, it will report the same revenue total it did on the 1^st of the new month.

The theory or justification for allowing this sloppy mix of data to pass for an actionable report is this: the left over transaction revenue is probably equal to the incomplete transaction revenue. So on-balance the report is probably pretty accurate.

And in some cases that’s probably true.

Unless your business has seasonality. Or product line changes. Or sales/promotions by either you or your competitors. Or impact from weather or other external major events. Or unless you add, delete, or modify your keywords. Or change your bids. Or test new ad-creative. Or work on your landing pages. Or in any other way change anything that might impact your customers, market, business, campaigns, or website.

Depending upon the degree to which you do any/all of those things, you’re going to need to take these reports with a proportionately large grain of salt.

• If you’re working aggressively on your PPC campaigns, trimming keywords (because the reports you now know you maybe shouldn’t trust told you that they weren’t making you any money – ah the irony), narrowing match types, and lowering bids, for example, you’ll see an exaggerated improvement in the following period as left-over revenue continues to come in from your old running keywords, higher-priced bids, and broader matches.

---

• Seasonality such as that we experience with a number of major apparel manufacturers, causes an artificial boost in some campaigns as a new season kicks in, as sales pickup aggressively from all the pre-season shopping that went on in the preceeding weeks. Another side effect from the current reporting method is that we don’t gain accurate visibility into when starting pre-season advertising is a good idea. With true-attribution we’d know.

---

• As promotions or seasons end, you get the opposite effect with exaggerated swings down. Now fewer latent conversion occur due to pricing or inventory issues, making a lot more of last periods click-cost become a lot less profitable (and perhaps even pushing into the red) more than any report will show.

Why Not Do It Right?

First let’s define who we’re talking about. As far as I know, everybody creates their paid search reports using the erroneous methods described above. Google, Yahoo, MSN, Google Analytics, IndexTools, and Omniture SiteCatalyst/SearchCenter are the ones I’m familiar with. Please leave comments if you know of any package that can properly relate clicks to revenue.

The obvious question is why. Why don’t paid search reports match clicks to their resulting revenue? Why doesn’t last month’s report include only the clicks that took place during that month and the sales that resulted from those clicks?

The answer seems to just be that this is how it’s always been done. It’s undoubtedly easier computationally. There are interface issues to doing it the way I’m suggesting.

Since the same software doing this reporting (the engines or your analytics package) knows both the average number of days for your latent conversions and the number of days for which they’re configured to allocate sales (as well as their allocation rules relative to multiple visits, but we’re leaving that complexity out of this discussion) a ‘last month’ report could include a simple flag disclaiming that ‘Report Incomplete – Additional Revenue Expected for 17 more Days’ or whatever.

Using accounting terms, it’s time paid search reporting moved from a cash to an accrual basis.

I don’t know why. I’ll see if I can’t get some folks from the engines or analytics vendors who might be willing to talk about this ‘on the record’ for a future post.

There are a number of factors which limit the accuracy of typical PPC Expense/Revenue reports.

• Deleted Cookies or other internet issues – Everyone doing business online has learned that all tracking has some significant limitations, which impact accuracy from a little (5% is considered typical) to a lot (cookie deletion rates have been reported in excess of 40%).

---

• Offline Conversions – Many visits that start with a paid search click end with a purchase in-store, via phone, or at home from a different computer. These cross-channel transactions typically aren’t tracked in the paid search reports.

---

• Attribution – Many visitors come to your site several times before they purchase, and use different tracked channels for each visit – once via paid search, another via email, another by typing in the URL. Which visit gets the credit for the sale depends on the capabilities of the system and in some cases the revenue attribution option is set in your analytics package. In any case, usually only one click gets the credit which means other do not.

---

• Latency – the fact that many people buy on a different day than their click, and paid search reports don’t accurately match clicks-to-purchase beyond the boundaries of the report time frame.

Each of these were mentioned in the comments of our Friday Quiz post question about problems with paid search reporting. And each definitely contributes to PPC report inaccuracy.

Deleted cookies became a visible issue most recently when Comscore reported that nearly 1/3 of all visitors removed cookies (first and third party) at least once per month. (More on the story here, and here.) While if true this behavior would overstate unique visitors (the metric in discussion in the report) it would cause PPC revenue attribution to be understated – as cookies are the most common way analytics packages and search networks connect visitors to their later purchases.

Offline conversions are an obvious and well discussed topic. This fact also under-states PPC revenue and attributions. There are inceasing ways to connect revenues from some channels (like phone orders) back to the PPC click that caused them, and via surveys and other methods many multi-channel retailers are learning how to at least factor offline sales back into their understanding of their paid search results.

Attribution is another systemic issue – people visit multiple times via multiple sources and at some point you (or your reporting software) has to decide which visit source gets the revenue credit for an eventual sale, if cookies or offline didn’t make it impossible to track in the first place! (Some very thoughtful posts here, here, and here.) The industry default is ‘last click’ but some analytics packages (such as SiteCatalyst and Coremetrics) provide control over that. (related post here)

What I Meant Was

These three issues – cookie issues, offline conversions, and attribution – are reasonably well understood and thereby can be taken into account by analysts and marketers. In my experience that’s not true for the issues we’re calling purchase latency, and that was the issue meant to be the ‘answer’ to our quiz.

A paid search report for a given time period – I used December 2007 as the example in the question – shows the PPC clicks count and expense during that period, and the sales from PPC (based on the selected allocation method) during that period. But any connection between those click and that revenue is in effect a coincident of time.

The fact is that every such PPC report is filled with revenue from left-0ver transactions (whose clicks took place before the reporting period) and what we’ll call incomplete transactions (whose revenue at least partially might come after the reporting period). Any hope of even approximate accuracy is based on the idea that there is a consistency over time for those keywords, those clicks, and those items across the time periods before and after the report. That’s not a good hope to base tens of thousands of dollars (or more) of spending decisions.

Our Winner

So I’m naming Tim Schaeffer the winner of our first Friday Quiz and a $50 iTunes Gift Certificate. Tim posted first, and concisely described the problem as:

If your product / service has a long purchase cycle ( 2 weeks), if someone clicks on your ad Dec 22nd. but doesn’t buy until Jan 3rd then the Dec 2007 report would show the click but no sale … and Jan 2008 report would show a sale but 0 clicks (assuming you only had 1 click on the keyword or just for logic sake).

More To Follow

In the next post I’ll share a lot more thoughts about this issue of conversion latency and it’s impact on PPC reporting.

Thanks to everyone for visiting and particularly those who participated in the quiz. Look for more on future Fridays.

Our friends and partners at Omniture announced Thursday that they’ve completed their acquisition of Visual Sciences. This is a huge event for the company and the industry. Congratulations are due on both the vision and execution.

The official press release makes clear (for the first time to my knowledge) that their intent is to migrate HBX customers over to SiteCatalyst. For the time being the product will be renamed SiteCatalyst HBX, which the release says will “continue to be supported until the key features have been integrated into Omniture SiteCatalyst.”

From my experience, that was done in 2002.

But the transition/migration is still a very large project since the site tags are incompatible (at least they have been until now) and more importantly SiteCatalyst ‘Classic’ has infinitely more capabilities and tagging requirements and options.

More importantly, the user interfaces and everything else about the two programs will require just about complete retraining of every user. The resistance will be high and the migration, if unprompted, would take a long time. It looks like Omniture is being wisely aggressive with events, training sessions, and a general gung-ho attitude.

I presume they did this acquisition for market share purposes, and who needs two incompatible code bases and two sets of training and customer service issues to chase the same market?

But are they the same market? Only partially.

It would be great to see Omniture use this opportunity to build a true mid-market product, both in recognition of market realities and in service of their higher end market and broad optimization platform.

Here’s what I would do:

• Rename HBX back to HitBox. The HBX rename was an attempt to upscale a downscale product. SiteCatalyst HitBox is a recognized, respectable and mid-market sounding name.

---

• Retool SiteCatalyst HitBox as the lite, mid-market version of SiteCatalyst Pro (you know they’re not going to call it Classic). Give it a streamlined UI and strip out all the super-power-stuff that many won’t ever use. Enable a one-button upgrade to Pro.

---

• Include free ‘taste’ features from Pro: Three Alerts. Two Classifications on Product, Pages, and Campaign Tracking Codes. A|B Comparisons on up to two metrics. Five Dashboards. Five Bookmarks. Two Targets. Three eVars. Two Correlations. Three Calculated Metrics. And killer video tutorials showing how and why to use each one of them.

---

• Price SiteCatalyst HitBox very aggressively. I’d suggest about ¼ of the cost of Pro with a minimum of $3000 per year. Leave the free stuff to Google and Microsoft, but provide serious analytics and more importantly an upgrade path for a price anyone actually doing business online can afford.

Here’s why:

• They need it anyway. The vast majority of SiteCatalyst users and more importantly prospective users in existing client companies can’t handle or digest the ‘Pro’ feature set and UI. This will certainly be true of HBX users who suddenly face 2X or 3X and controls, options, menus, etc. The company rightly prides itself on depth of adoption inside client companies, and could probably double it with an approachable UI version. (No economic loss here, Pro companies just get the option to let some users log into the HitBox version) And ultimately perhaps 25% of HitBox users migrate into Pro users or cross-product suite users over time.

---

• There is a mid-market. The distance between Google Analytics and SiteCatalyst is huge, in price and capabilities. Online businesses in the $1M to $10M segment haven’t been your target but there’s a lot of them. I believe SiteCatalyst Pro is a justifiable investment for them, but most can’t be convinced to make the leap. A way to ease into it – both financially and by tasting what large benefits the higher end could provide, would be very successful.

---

• Tomorrows Analysts are GA users today. From blogs to startups millions are getting used to GA, and will migrate to new and large companies over time and say ‘it’s good enough’ to bosses who don’t know better. Push GA down market with a legitimate mid-market option. Let all those folks in the $1-$10M companies get hooked on power and convince their companies to upgrade.

---

• Sell The Blades. Omniture has built an impressive suite of products, and beyond HBX this acquisition provides a bunch more. As these mature and continue to be integrated a broader set of customers can only get pulled either across the line or up into the Pro line.

---

• Data. It’s clear that data is the oil of the 21^st century. Anyone who can sit on top of lots of it and knows how to get at it (or will let someone pay for that privilege) will be fat and happy for a long long time. Taking another 25% to 50% of GA’s market share is like annexing Kuwait in terms of traffic, behavioral and transactional data. Given this, the other four reasons hardly matter.

Why not:

• Fear. The main reason not to do this may be a fear that less will be good enough, thereby spawning ‘downgrades’ and lower ticket new sales. I think it’s a legitimate worry. It should be taken as a challenge to prove the value of the high end and make SiteCatalyst ‘Pro’ approachable via a better UI and an experience that enables more customers to take full advantage. The value is there it just needs to be made more accessible.

---

• Resources. Yes it’s an additional project on a list that must be miles long. For all the above reasons, with the ultimately limited amount of information I have from the outside, I think it’s worth it.

Coda

I should disclaim all of the above with the information that Commerce360 is an Omniture Platinum Partner, I’m on their Customer Advisory Council, and I’ve been over time briefed and subject to NDA’s on future Omniture products. I’ve never had a discussion with anyone there about the HBX merger or HBX products or anything relevant to this post in terms of their product line directions or these suggestions. No animals were harmed in the development of this blog post.

Part of the reasons for this post is that I’ve seen this movie before. In the early 1990’s I had a lot of involvement and ultimately worked at Aldus, the PageMaker company when they were the hot young public software player in the darling space, revenues shooting past $100M and acquisitions rapidly growing the product line. They also had a huge industry player (Microsoft) giving away competitive programs (via the then new Office bundle) and one remaining legitimate competitor to each of their main high-end programs.

There are a lot of dis-similarities here too, but Aldus wound up acquired by Adobe for a number of reasons you would have never predicted in 1992.

I also want to be clear (if it isn’t already) that I’m a huge Omniture fan. I think SiteCatalyst and SearchCenter are the only reasonable serious analytics choice in terms of the features they provide. They’re far from perfect, but I’m pretty well versed in each of their competitors and would/do strongly suggest to anyone that picking any of them is, based on my experience with current versions, a compromise at best and a bad (or very bad) decision at worst.

This a great but complicated and important time for Omniture. They’ve been navigating it brilliantly, and I’m rooting for their continued success.

Paid search reports are wrong. Inaccurate. Mis-leading.
Do not mean what you think they mean.

Do you know why?

First right answer in the comments wins a $50 iTunes gift certificate.

Here’s the statement that must be explained:

“A December 2007 PPC Expense/Revenue report pulled from either the paid search engine management consoles or your web analytics software will not accurately report how many units were sold or how much revenue was generated by December 2007 paid search spend because…”

Clues: Neither Javascript nor Cookies are involved.

Rules: Partial answers probably won’t be accepted. I’m the judge. Via comments you may all be the jury. Or the unruly crowd. But the Judge has final say. If not accurate winner, I may award the prize to the most humorous. No accounting firms or state laws involved. Commerce360 employees and people who’ve heard me tell this story are ineligible.

The Winner and The Answer will be posted on Monday.

When looking at the stats for one ad-group within a campaign, isn’t it likely that you’ll next want to see the stats from the following ad-group in the campaign? Or the previous one? Or dive down to the keyword level? And then move to the keywords for the next ad-group. Or move up from ad-group to see those same stats at the Campaign level?

While looking at just about any paid search analytics report it should be easy to move up, down, left, or right.

It isn’t.

Adwords

In the Google Adwords interface, you can go down in a pretty natural manner – by clicking on a campaign or ad-group name. To go up requires a click in the breadcrumb nav bar. There is no next or previous, you have to go up then back down.

This might sound trivial. Until you do it for an hour or two. Or 15 times a day. It’s a massive time waste and more importantly ultimately prevents analysis. At some point it’s just not worth it to look over those last few ad-groups, or check that new theory on the two campaigns you’ve already seen.

Adwords Editor

In Google’s fantastic Adword Editor, the problem is largely solved by the 3-pane design. No matter what you’re looking at you can switch very quickly to another campaign or ad-group. And the tabs across the top make it simple and quick to go from keywords to negatives to text-ads.

AE isn’t primarily an analytics package, so it doesn’t really solve this problem, but it’s a great example of how elegance and speed make you both want to use a tool and use it a lot more. You can jump into AE, look at things, make some changes, and even take the extra step of having to submit them (since you’re working off-line) and it feels efficient.

Omniture SiteCatalyst / SearchCenter

In Omniture SiteCatalyst the problem is exacerbated because there can be a lot more data (especially if you use a lot of calculated metrics as we do), the interface is way heavier, and let’s face it – SiteCatalyst is generally to painfully slow.

In version 13.5 SiteCatalyst added a very handy drill-down feature to SearchCenter data that makes it pretty simple and sometimes even quick to drill from engine to campaign to ad-group to keyword.

Unfortunately, moving back up the hierarchy is neither elegant nor fast, and there remains no way to go sideways other than going all the way back up (which basically means starting over).

In some cases the ‘Select Ad Group’ or ‘Select Campaign’ dialog box does allow a semi-direct way to switch from one to another, but the cost is three terribly slow transitions. (One for the dialog itself to appear, another to execute a narrowing search, and a final one to draw the resulting report – I just did one and it took a full minute to complete these three steps just to shift from one campaign to another).

Conclusion

I spent nearly all of today navigating the reports described above. With a nice little 4-way navigation button, or if the 3-pane design were moved into Adwords and adopted by Omniture, I’m pretty sure I could have been done by lunch.