Data Driven Design: Using Web Analytics to Improve Information Architecture / Andrea Wiggins

Saturday, March 24, 2007

Web analytics= think WebTrends although there are other tools out there.

Web analytics can be used to:

• quantify user experience audits
• identify key performance indicators
• compare over time with annual audits

Limitations:
Most tools are not designed to capture Rich Internet Applications (RIA); user may stay on one "page" while interacting with content.

Spiders ruin user data

• block out with robots.txt—prevent from looking at logs
• can also identify spiders by looking at speed of visits from single user

Types of Data:

Ratio of new to returning visitors

• think about context
• track over time and track with cross-channel marketing
• consider the effect of timeouts
  

Median visit length

• is closer to reality than an average visit length
• can indicate depth and breadth of visit—are they digging deep or are they hopping around?

Clicktru rates for clickable graphics--requires additional programming (we've done this for our Featured Connections).

Response time--be sure to check at peak load time

Server errors

• Monitor 500 server errors, which is where our server has the problem
• Try to identify how the user got to the 404 server errors
• Combined, hits to server errors should be < .5%

Action items:

• Look at those dang 404 errors that show up in WebTrends more closely. Can we track where they are coming from?
• Look at Crazy Egg analytics tool
• Look at “leakage points”—where did users bail out of the website. Do they make sense?

Links for More Info:

• Slide show presentation from SlideShare

Using Search Analytics to Diagnose What’s Ailing Your IA / Rich Wiggins and Louis Rosenfeld

Saturday, March 24, 2007

Wiggins’ emphasis was best bets within search results. Rosenfeld spoke more generally about identifying problems from search logs.

Practicalities:

• Zipf curve (long tail/short head) applies to search log—many users have unique needs
• Look at top searches, and then dip down into the unique ones. Don’t treat all the searches as equal. Could look at top 50% of all searches, for instance.
• Consider seasonality (by season, day, even hour). Some needs are higher by season. Could promote that content accordingly.
• Capture search logs to SQL database to then process. Can dump relevant fields into Excel and then evaluate.
• Use IP with time stamp to surmise single user.

Ways to Use Search Logs:

• Look at most common unique queries; are there patterns?
• Test common queries to see what results look like.
• Look for null results.
• Look for too large results.
• Can grow content to satisfy searches (ex: Netflix did this in response to “yoga” searches)
• Look at improving search entry, results, and/or algorithm
• Combine with field study (ex: L.L.Bean saw users starting with catalog, then taking SKU to web—answered why users were searching for SKU)
• Fixing a trend seen in long tail could help many.
• Look for time variations; respond by positioning Best Bets or guides seasonally.
• Add tools for results page (i.e. options for broadening/narrowing)—this moves advanced search options from search page to results page
• Best bets as not the final answer; still should monkey with relevance ranking. (ex: rank company names higher if that’s what users search)
• Consider a best bets index rather than/in addition to a site index. See MSU A-Z index as example of using common queries as a best bets index. Site index uis difficult to build. Do you make it comprehensive? Selective?
• Look for the page the user is searching from to identify failure points.
• Look at top pages found through search; how can these be easier to find in navigation?
• When cleaning up site, start with what people want—rather than complete evaluation of content.
• Look at “tone” in search (technical or popular; specificity; acronyms; plural) to help create labels.
• Cluster queries to see parent/childs; look for possible metadata fields and contents for those.
• Sample the long tail (tends to be more research oriented)
• Compare spikes (proper names, companies) and compare with editorial content; identify future stories. (ex: Financial Times has done this)

Links for More Info:

• Slideshow presentation and podcast