• by Judah Phillips , November 17, 2009

I've often said that not all Web analytics tools were created equally.  Each tool has various nuances that make it either very appropriate or less than effective for your business.  While looking at simple lists of features, hearing the sales-speak, and watching product demos is certainly helpful for assessing tool capabilities, limiting yourself to only those activities is never wholly sufficient to put the "tool decision" in context.

 

 A practitioner's experience, an industry analysts' viewpoint, and a comprehensive proof-of-concept are absolutely necessary to help frame a tool's functionality and evaluate it against business need.  During my career in Web analytics, I've found certain functionalities to be absolutely priceless for creating, reporting, investigating, and analyzing the site data necessary to derive those often cited but hard-to-realize "insights."  Some of the features I think you need include:

Ad-hoc querying capabilities in one tool.  The ability to work within a single interface to explore data is invaluable in Web analytics.  Many "enterprise" analytics tools do not allow for the deep drilldown, filtering, and dimension crossing necessary to adequately segment traffic in one interface.  The analyst can often be scattered across multiple tools to generate the data necessary to begin to evaluate the answer to a business question.  That's suboptimal, especially when tools can operate off different data models and use different algorithms.

Hybrid data collection.  While most of the world has moved, more or less, to JavaScript page tags as a data collection standard, there's still value in being able to parse and index log files.  Remember, many different applications generate log files, not just web servers, and being able to bring application-specific data that contains countable name/value relationships into your Web analytics tool can be invaluable to your business. 

Integration capabilities.  While there is value in looking at the basic dimensions and measures in Web analytics, such as views, visits, and visitors, referrers, and time spent, there is a world of data well beyond the simple stuff.  The best Web analytics tools allow for data integration with both internal and external data sources, such as CRM systems and third-party vendors like ESPs.  Evaluate carefully your needs in this area; it's where the big ROI will be found.

Browser-Based. There are actually analytics tools that require proprietary desktop installations or operate outside of a browser environment.  While I understand why these tools were created in this manner (security, differentiation), it becomes very difficult to extend these tools across a large user population (if that is a concern).  So before you jump at what looks really cool and powerful in a demo, consider whether you can truly extend such tools to meet the needs of your user population.

Visitor-Level detail. Recently I was asked, "Why do you need visitor-level detail?"  The answers: "relationship marketing" and "deep segment comparison."  If you can link individual user behavior from marketing campaigns and trigger actions based on rules and events at the individual level, you will be able to do things your competitors just can't (until they catch on!).  From a Web analytics perspective, there are very few vendors who provide real visitor-level data access. 

Alerts.  I like alerts.  Alerts let you know when something has exceeded a predefined threshold and thus act as a call to arms to investigate the anomaly.  Several vendors offer alerting capabilities at various levels of sophistication.

Statistical Functions.  Harder to find in off-the-shelf analytics tools, statistical functions, such as calculating standard deviations, UCL/LCL, probability, and regression, can be helpful in putting the data into the right context and guide you to hard-to-discover answers.

Visualization Options.  For those who are less than comfortable looking at the numbers, insights can be instantly realized through data visualization.  Whether via trend lines, scatter plots, pie charts, histograms, spider charts, multidimensional visualizations and other eye candy, having the ability to process your data into these visualizations is something that will make data more consumable to your stakeholders.

Dynamic Report Generation.  I am not a big fan of simple reports.  I enjoy being able to dynamically add metrics on the fly and employ those ad-hoc querying capabilities I described earlier.  Too often, Web analytics reports are just that - reports.  By creating a dynamic experience within the report itself, insights that could remain undiscovered can be brought into the spotlight.

Once you assess a feature set against requirements, I strongly advise you to hire a sufficient number of analysts to use these functions to integrate the data until  you unlock value-generating insights.