Google Analytics, I Can’t Trust You

While Google Analytics is indeed free and includes some great features, I have one giant red flag issue with it: sampled data, AKA Fast Access Mode.

My experences with Google Analytics have led me to become disenchated with the tool for large data sets. When you drill down to the critical intersection of SEO data — which URL drove visits & transactions for which keywords — the data is sampled to such an extent that it is useless to base decisions on.

Why?

Sampling occurs when you add a secondary dimension to a report and the resulting data for that segment contains more than 500,000 visits. Sampling data allows Google Analytics to generate reports quickly for large data sets.

For example, I’ve seen a filtered report comparing 2 months in which every one of 300 URLs incresed 100% or decreased 100%. Every one of them. While not physically impossible I suppose, it’s just not realistic.

In addition, the yellow degree of accuracy numbers often indicate +/-15% to +/-89% accuracy. I find it very difficult to base important decisions on data that is so highly filtered and sampled. Here’s a fictional example obfuscated from a real GA account. Take a look at the third row, highlighted in red. GA reports 30,391 with a +/-17% degree of confidence. That +/-17% degree of confidence, represents a 10,333 swing in visits. So I’ve either driven 35,557 visits, 25,224 visits, or somewhere in between. With a swing that wide, I really have no idea whether I’ve increased visits to this page for this phrase. So how can I measure my success, let alone plan future optimizations?

Google’s help forums are filled with rants and pleas for Google to remove Fast Access Mode. data sampled to the point of ridiculous inaccuracy is worse than no data at all. I would much rather wait for GA to email me a report with the actual data, than access a useless sampled report quickly.

Yes it’s free, and yes it’s better than nothing … I think. But beware, you get what you (don’t) pay for. Take your analytics data with a nice big grain of salt, and look as closely for what it’s NOT telling you as for what it is.

Of course, if you’d like to PAY for unsampled reporting, Google just announced a new Google Analytics Premium service. How much does it cost? Does it solve all these problems? Do I want it? Can I trust it? Is it really worth the unknown cost? Don’t know. You have to fill out a faceless form and have their sales rep call you back. Really? Really.

It’s a smart move on Google’s part, I guess — get us hooked on the free but flawed product to make it theoretically easier to get us to convert to a paid analytics model instead of fixing the issues in the free version. I guess if you’re big enough to have the sampling problem you need to put on your big boy underwear and get an enterprise-level analytics package anyway. Still, it just cheeses me off the way it’s been handled.