Welcome to Standing Stone Designs

Absolut Urchin: The Best Keeps Getting Better

OK, so you bought web analysis software a few years ago after looking closely at the competition. Maybe you read about Urchin 4 decided Urchin was the best choice (we always said you were a smart cookie). But should you upgrade? Isn't good enough, well, good enough? Not if it keeps getting better. Since 2000, I have tested several web metrics analysis programs on several occasions, concluding in 2003 that Urchin 4 was the best product on the market (you can read my earlier reviews in New Architect magazine: Know Your Visitors and Traffic Analysis for Busy Sites and at ProSoundWeb). Since then I put Urchin 5 to the test, concluding that it's definitely a wise move to upgrade from Urchin 4 or, for that matter, any other application you're currently using.

In previous testing, I demonstrated that I demonstrated how Urchin is the fastest, most accurate web metrics analysis tool available, including features which made Urchin the best choice for any family of web sites, from small sites to very large data centers, including sites like Prosoundweb.com (where I am webmaster) which combine to average over 90,000 unique visitors turning over 2.5 million pages each month, or here at Standing Stone Designs, where we host 2 dozen small sites. As our demands have only grown, we also upgraded from Urchin 4 to Urchin 5, and I have been watching the performance and features of the new software ever since. For example, with the launch of the new ProSoundWeb forums in April 2004, I was really able to put Urchin to the test.

Urchin 5 continues the product's tradition of extremely fast data processing, crunching a 2 gigabyte log in under 20 minutes compared with several hours for higher-priced Web Trends. But speed is not all; Urchin has added several valuable features to their product, enhancing both performance and usability. These include the improved UTM tracking module, SVG graphics, and a much-improved user interface.

Urchin Tracking Module (Traffic Monitor) Enhancements

At the heart of Urchin is the UTM or Urchin Tracking Module, also called the Urchin Traffic Monitor, which virtually eliminates inaccurate stats caused by proxy servers and caches. Urchin first offered UTM

Fig. 1: May 2004 Visitors by Day, Week of May 23-29 Graph, SVG style, showing first time and repeat unique visitors, also showing user interface with Reports and Date Range menus (click on image for a larger image)
technology in Urchin 4, a bit of JavaScript which forces a query to the web server, by-passing proxy servers, to ascertain that a request has actually been made to your server. Why are proxy servers a problem? Because many ISPs (Internet Service Providers, including AOL and EarthLink), in their on-going effort to improve delivery speed to their clients, cache frequently-requested pages for up to a day on so-called proxy servers to speed delivery (saving the bandwidth and time involved in requesting the page directly from more remote servers). This may cause the page to be outdated when you receive it (though there are ways of dealing with that issue, separate from web metrics analysis), but germaine to our discussion, your web server will never know that a request was made because the proxy server delivered the page and the request was never seen by your server.

A related problem appears when ISPs (again, like AOL) dynamically assign IP addresses to their users. (What is an IP address? It is the numeric sequence used by networked computers to identify each computer; humans think in terms of domain names like standingstonedesigns.com, but computers think in terms of numbers like 216.129.136.59.) Traditionally, a common way to identify unique visitors is by IP address, but with IP addresses assigned on-the-fly by ISPs, a single user may be seen by your server as several IP addresses and thus be interpreted as several unique visitors, when in fact they are not. How does the UTM deal with this? It sets a first-party cookie (that is, a cookie associated with the host domain—your domain—as opposed to a third-party cookie, which is associated with another domain and which may therefore be seen as a security risk) which is recognized by the system and is used in place of the IP address for visitor tracking. By simultaneously defeating both proxy caches and dynamic IP addressing, UTM offers the most accurate tracking available.

OK, that's cool, but all of the above was present in UTM in Urchin 4; why upgrade to the latest version? Because Urchin has wisely extended the functionality of the cookie whilst keeping it small and unintrusive.

Fig. 2: May 2004 Screen Resolution Graph, Pie Chart style (click on image for a larger image)
These advantages include clickthrough path tracking (that is, the trail visitors follow through your site from their entry point on), and providing data on Client Parameters, such as screen resolution. Personally, I find this latter extremely useful, especially as we consider any redesign or new site or page designs at Standing Stone Designs: to successfully optimize the appearance and function of the site, we want to know not only what browsers visitors are using, but other client parameters such as monitor resolution, whether Java is enabled, what JavaScript version is running, and what OS platform the user is running. For instance, when we decide the basic width of a web page layout, one very useful bit of info is how many visitors are viewing our site at 1024x768 (currently over 53%), 800x600 (over 20%), or 640x480 (the mid-90s standard; less than .5% of our visitors). Thus, since we know that 99% of our visitors are viewing the site at at least 800x600, it's safe to use 800 as our standard width (in fact, 79% of our visitors use screen resolution above 800x600, but 20% is too large a segment to push the margins any higher). In any event, we now can analyze this data thanks to Urchin's enhanced UTM.

How does UTM work? In addition to the cookie described above, the UTM JavaScript and an associated tiny gif are stored at your web root; you only need to slightly modify your web server config file (a breeze with Apache), and tell Urchin itself that you are using UTM tracking, and you have the most accurate visitor tracking mechanism available. It is therefore not only very powerful but extremely easy to work with.

Enhanced SVG Graphics

Fig. 3: April 2004 Pageviews Graph, SVG style (note that mouseover on Tues Apr 06 outputs the total pageviews for that day in the left column and the day, date, and pageviews above the graph - click on image for a larger image)
OK, enough technical mumbo-jumbo, let's have a look at some of the feature enhancements of Urchin 5. Most notable are the vastly improved graphics; if you check my review from 3 years ago for what was then Web Techniques and is now New Architect magazine, you will see that one area where I gave WebTrends much higher marks than Urchin was in their graphics: Urchin's graphics were barebones, much less suited to a boardroom presentation than WebTrends'. Of course, there was a cost behind WebTrends' superior graphics: each graphic was built when the report was generated, and therefore had to be stored on a server, taking up both processing time to build and storage space to store the images.

Fig. 4: April 2004 Pageviews Graph, SVG Line style (click on image for a larger image)
Urchin, on the other hand, was designed to build the graphics (and the reports themselves) on the fly, whenever the data is queried. Thus, instead of storing static HTML pages and associated images, Urchin 3.3 (the version I tested three years ago) produced only three lean data files per site per month, applying the data to templates to produce dynamic HTML/JavaScript report pages. The JavaScript handled the final calculations for averages and other results, producing reports for any user-specified date range without creating bulky graphics files. For large data centers hosting 100s or 1000s of sites, large sites generating a multitude of different reports (the PSW case), or sites where multiple users frequently pull up a wide range of reports based on user-defined periods (rather than simply daily, weekly, or monthly), Urchin had a clear advantage in much faster processing, not only of the original logs, but also for generating each report on the spot (which takes only seconds). But to produce fancy graphics to show to the board of directors or customers, at the time the best option was to export the Urchin data to Excel and produce new graphics.

No longer. With v. 5, Urchin has introduced SVG (Scalable Vector Graphic) graphics with a full range of pie charts, bar charts, and line graphs with 3D colors and shadows, highly suitable for export to PowerPoint or Word.

Fig. 5: Pie Chart View 2: April 2004 Requested Pages Graph (click on image for a larger image)

Fig. 6: Pie Chart View 3: April 2004 Requested Pages Graph (click on image for a larger image)
Thus we can now present much sharper graphics to our customers (those good folks who pay for the ad space on the Huge Universer (ProSoundWeb's parent site) sites and therefore allow us to keep bringing our content to you). Among the very active clients we host at Standing Stone Designs is the Blackstone Valley Chamber of Commerce, and several times now I've used output from Urchin's graphics in web metrics reports to their Board of Directors. I also distribute reports as needed for any of my other clients. Here are some examples of Urchin 5's graphics:

Figures 3 and 4 show two alternate views of the Pageviews graph, one as an SVG bar graph and the other as an SVG line graph (there is a third option, the standard bar graph). In each case, you can mouseover a given day to output that day's totals. Now, let's take a look at the new pie charts Urchin offers, this time looking at Requested Pages. Fig. 5 shows the the top 10 pages as a pie chart, with mouseover on the second largest pie segment showing the single most requested page and outputting the data below the chart and highlighting the corresponding line in the detail section in yellow. Clicking on any given segment will split that segment out from the pie, as shown in Fig. 5 with four segments broken away, showing the top four most requested pages.

User Interface Improvements

Urchin has also streamlined their user interfaces, one for administering Urchin (thus enhancing the administrative interface), the other for the reports themselves (thus enhancing the report interface). The most obvious example is that now, on every page of every report, you will find an interactive calendar which allows you to quickly select any date range you want to pull a report. Urchin always offered this functionality, but it used to require several additional steps to define the date range. Now it's extremely easy to select any date range against which you'd like to generate a report. And since the reports are created on-the-fly in mere seconds, you don't need to store these reports in static form. This surpasses WebTrends and all other web metrics programs I've used over the past several years.

Urchin features multilingual reporting, offering English, French, German, Spanish, and Japanese (including double-byte character support for Japanese). The user interfaces also feature multiple filtering options (which can be quite granular if you wish), sorting, time series analysis, export capabilities in tab-separated text, Word, Excel, and a print-friendly view, and built-in help with a glossary and links back to the knowledge base on the Urchin website. Reports include:

Fig. 7: Platform/Browser Combos - PSW (click on image for a larger image)

Fig. 8: Platform/Browser Combos - MARSH (click on image for a larger image)

the overall summary page, which is concise and provides an instant thumbnail of activity in terms of sessions, hits, pageviews, and bytes transferred grouped in totals, average by day, and average by session
Unique visitors and sessions, visitors and sessions by day, visitor loyalty, session frequency, and traffic summary
Requested pages, downloads, page query terms, posted forms, status and errors, hits and bytes downloaded per directory, file, and file type
Entrance pages, exit pages, click paths, click to and click from, length of pageview, depth of session, and length of session
Referrals, search terms, search engines, and referral errors
Domains, Countries, and IP addresses
Browsers (showing sessions and percentages and sorted first by main type such as IE, then by subtype such as all versions of IE), platforms (displaying the same type of data as browsers), robots, and, very useful, a "combos by session" which aggregates the platform/browser combinations accessing your site (so that, at a glance, you can see for instance how many sessions in a given date range combined Windows XP with IE 6 vs. Mac OS X with Safari 125.7; as shown in fig. 7 and 8, this can make quite a difference as the percentage of Mac / Safari users is much higher on our MARSH forums than on the PSW website in general, reflecting the higher use of Mac's by recording artists).

Over three years ago I became convinced that Urchin offered the fastest, most powerful web metrics package for data centers and other large websites. With Urchin 4, I found that Urchin had surpassed industry leader WebTrends in most areas for small and mid-sized websites as well. Today I am convinced that Urchin offers "absolutly" the finest web metrics software available, at a very compelling price (less than one-half the cost of WebTrends).

To learn more about Urchin, visit the Urchin website.