Using Search Analytics to Diagnose What’s Ailing Your IA / Rich Wiggins and Louis Rosenfeld

Saturday, March 24, 2007

Wiggins’ emphasis was best bets within search results. Rosenfeld spoke more generally about identifying problems from search logs.

Practicalities:

• Zipf curve (long tail/short head) applies to search log—many users have unique needs
• Look at top searches, and then dip down into the unique ones. Don’t treat all the searches as equal. Could look at top 50% of all searches, for instance.
• Consider seasonality (by season, day, even hour). Some needs are higher by season. Could promote that content accordingly.
• Capture search logs to SQL database to then process. Can dump relevant fields into Excel and then evaluate.
• Use IP with time stamp to surmise single user.

Ways to Use Search Logs:

• Look at most common unique queries; are there patterns?
• Test common queries to see what results look like.
• Look for null results.
• Look for too large results.
• Can grow content to satisfy searches (ex: Netflix did this in response to “yoga” searches)
• Look at improving search entry, results, and/or algorithm
• Combine with field study (ex: L.L.Bean saw users starting with catalog, then taking SKU to web—answered why users were searching for SKU)
• Fixing a trend seen in long tail could help many.
• Look for time variations; respond by positioning Best Bets or guides seasonally.
• Add tools for results page (i.e. options for broadening/narrowing)—this moves advanced search options from search page to results page
• Best bets as not the final answer; still should monkey with relevance ranking. (ex: rank company names higher if that’s what users search)
• Consider a best bets index rather than/in addition to a site index. See MSU A-Z index as example of using common queries as a best bets index. Site index uis difficult to build. Do you make it comprehensive? Selective?
• Look for the page the user is searching from to identify failure points.
• Look at top pages found through search; how can these be easier to find in navigation?
• When cleaning up site, start with what people want—rather than complete evaluation of content.
• Look at “tone” in search (technical or popular; specificity; acronyms; plural) to help create labels.
• Cluster queries to see parent/childs; look for possible metadata fields and contents for those.
• Sample the long tail (tends to be more research oriented)
• Compare spikes (proper names, companies) and compare with editorial content; identify future stories. (ex: Financial Times has done this)

Links for More Info:

• Slideshow presentation and podcast