User-Centered Design, Inc., Ashburn, va 20147


Download 55.41 Kb.
NameUser-Centered Design, Inc., Ashburn, va 20147
page1/4
A typeUser
manual-guide.com > manual > User
  1   2   3   4

Limitations of Server Log Files for Usability Analysis


Karl Groves

User-Centered Design, Inc., Ashburn, VA 20147

kgroves@user-centereddesign.com

Limitations of Server Log Files for Usability Analysis

Table of Contents


Limitations of Server Log Files for Usability Analysis 1

Limitations of Server Log Files for Usability Analysis 2

Table of Contents 2

Introduction 2

What is a Server Log File? 3

Analysis Tools 4

How the Web Works 4

The HTTP Protocol 4

Caching 5

Caching Wreaks Havoc on Statistics 8

Caching's Effect on Margin-of-Error 8

Usability Data Cannot Be Found In Log Files 8

Usability Data: Who Is Coming To Your Site? 8

Usability Data: Can Log Files Indicate How People Navigate on a Site? 10

Usability Data: What Element(s) Did the User Click? 11

Usability Data: How Long Did the User View the Page(s)? 11

Usability Data: Where Do Visitors Leave Your Site? 12

Stats Do Not Represent How Many Visits (or Visitors) The Site Had 13

Stats Cannot Tell You Where the Visitors Came From or Where They Entered 13

Stats Cannot Measure Users' Online Success 13

"Tracking Usage Trends" Provides No Useful Data 15

The Special Case of AOL 16

Conclusion - Server Log Analysis Is an Unreliable Tool for Usability 18

Introduction


One of the challenges faced most often by those of us in the field of usability is finding good data about user behavior quickly, accurately, and, in most cases, cheaply. In an environment where many stakeholders question the return on investment in usability, some in the industry have developed interesting ideas aimed at gathering user data. One such idea is the analysis of server log files to gather information about user behavior. On the surface, it is easy to understand the gravitation towards server logs: they're supposedly a data source which portrays what people are doing on a site. Server logs supposedly show what people click on, what pages they view, and how they get there.
Unfortunately, practitioners who espouse such methods seem to lack important technical knowledge regarding the nature of the web, the Hypertext Transfer Protocol (HTTP) and the process of caching within networks, proxies, ISPs, and browsers. These technical details greatly limit the types, and quality of information that can be retrieved from

server logs.
In addition to the technical limitations of server log file analysis, without information regarding exactly what the user expects to find and why they make the choices they make, there's no way for us to know whether they were successful in their quest and whether that quest was satisfying - and that, ultimately, is the usability information we seek.
Server log files are inappropriate for gathering usability data. They are meant to provide server administrators data about behavior of the server, not the behavior of the user. The log file is a flat file containing technical information about request after request for files on the server. Log file analysis tools merely assemble them in a conjecture-based format aimed at providing insight into user behavior. In the commentary below, I will explain why the nature of the web, the HTTP Protocol, the browser, and human behavior make it impossible to derive little meaningful usability data from server logs.
First, some of the technical background information is needed.

What is a Server Log File?


Server traffic logs are files generated by the server in order to provide data about requests to the server for data. When a computer connects to a site, the computer, browser, and network will deliver some data to the site's server itself to create a record that a file was requested. Here's what an entry into a log file looks like:

86.42.132.114 - - [31/Oct/2005:18:15:16 -0500] "GET /styles/style.css HTTP/1.1" 200 5194 "http://www.example.com/links/links.php?cat=css" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7"

The format above is from an Apache log. Depending on the type of server the site is on, the log entries may look different. Thousands (or even hundreds of thousands) of entries such as the one above are placed into a plain text file, called the server log.

The above log entry includes the following information:

  1. IP Address of the requesting computer: 86.42.132.114 This is not the user’s IP address, but rather the address of the machine they’ve connected to.

  2. Date & time of the request: [31/Oct/2005:18:15:16 -0500] That's October 31, 2005 at 6:15:16pm and the time zone is 5 hours behind GMT, which is Eastern Standard Time in the USA (this is because the server is in that time zone, not the user.)

  3. The full HTTP request: "GET /styles/style.css HTTP/1.1"

    1. Request method: GET

    2. Requested file: /styles/style.css

    3. HTTP Protocol version: HTTP/1.1

  4. HTTP Response: 200 This means the request was ok.

  5. Response size: 5194 bytes This is the size of the file that was returned.

  6. Referring document: http://www.example.com/links/links.php?cat=css The links.php file is referring its embedded style sheet.

  7. User-Agent String (Browser & Operating system information): "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7" The user’s computer is using Firefox browser on Windows XP and their language is set to English - US.

Of all of the information in the log entry, only the time & date, HTTP Request, and the response information should be regarded as accurate. The IP address, referrer, and user-agent string should be regarded as unreliable, as it can be faked in some way by the user. For example, Netscape 8 gives the user the option to identify itself as Internet Explorer during the setup process and many other browsers supply this ability in their “Options” or “Preferences” menus.

Analysis Tools


Many organizations use an analysis program to parse server log files so that they're much easier to understand. Imagine trying to cull anything of value from a huge text file of entries like the one above when hundreds of thousands (or even millions) of entries are present in the log file! Essentially, the analysis tool treats the log file as a flat-file database, processes it, and generates the "statistics" that are discussed throughout the rest of this commentary. In other words, “Web Analytics” software does little more than provide its own interpretation on data contained in the log file which, as has been stated above, could be at least partially faked.
  1   2   3   4

Share in:

Related:

User-Centered Design, Inc., Ashburn, va 20147 iconUser Centered Design

User-Centered Design, Inc., Ashburn, va 20147 iconUser-Centered Design Process

User-Centered Design, Inc., Ashburn, va 20147 iconUser Interface Design Process Obstacles -usability -human Characteristics In Design

User-Centered Design, Inc., Ashburn, va 20147 iconDesign manager and ui architect with broad history in user experience...

User-Centered Design, Inc., Ashburn, va 20147 iconOf Person-Centered Service/Care Planning

User-Centered Design, Inc., Ashburn, va 20147 iconFm radio transmitters. With its unique large lcd design and membrane...

User-Centered Design, Inc., Ashburn, va 20147 iconUser Interaction and Experience Design

User-Centered Design, Inc., Ashburn, va 20147 iconUser Interface Design Engineer for over 15 years

User-Centered Design, Inc., Ashburn, va 20147 iconEnd user license agreement for solutions design bv software

User-Centered Design, Inc., Ashburn, va 20147 iconEnd user license agreement for solutions design bv software

User-Centered Design, Inc., Ashburn, va 20147 iconUser Interface Design and usability expertise (with Accessibility practices Section 508)

User-Centered Design, Inc., Ashburn, va 20147 iconAbstract Using the case study of Honda Cars, this paper confronts...

User-Centered Design, Inc., Ashburn, va 20147 iconProfile 9+ yrs of experience in ui / Ux / Website / Graphic / Logo...

User-Centered Design, Inc., Ashburn, va 20147 iconProfile: Very experienced multi-discipline Design Draughtsman and...

User-Centered Design, Inc., Ashburn, va 20147 iconAbstract design competitions has been one of the major tools for...

User-Centered Design, Inc., Ashburn, va 20147 iconPigeon Point™ bmr-avr-amcm Reference Design Board Management Reference...

User-Centered Design, Inc., Ashburn, va 20147 iconUser interface design and implementation; graphics, image processing...

User-Centered Design, Inc., Ashburn, va 20147 iconAbstract Because optical performance has increased so far with modern...

User-Centered Design, Inc., Ashburn, va 20147 iconGnat-note #1 Jan 31, 1993 Revised: April 8, 1993 Revised: April 12,...

User-Centered Design, Inc., Ashburn, va 20147 iconDesign Requirements




manual


When copying material provide a link © 2017
contacts
manual-guide.com
search