Keep a Local Copy of Google Analytics With More Precise Timestamp #measure

HOWTO: GA on Your Server

How to Save and Process Google Analytics Data Locally

When Jason Thompson asked his question about more precise visitor times as quoted by Avinash I thought about setting a custom variable with a JavaScript function.

There is another alternative with an additional benefit: store a copy of your Google Analytics locally on your server.

If you were to do this not only do you have a backup copy of your Google Analytics data, you can go back and reprocess the data if you have the technical capabilities.

Official Google Analytics Documentation

According to the official Google Analytics Documentation, Google Analytics does not track more precisely than to the hour.

That is, of course, not the whole story.

Google Analytics Hack

I highly recommend ‘Advanced Web Metrics with Google Analytics‘ by Brian Clifton, he provides solid instructions and details for many implementations of Google Analytics. I’m going to use either his examples, or modified versions of his examples.

Brian’s book provides an example for a traditional snippet, the Google Code section of Google Analytics provides pretty clear instructions:

_gaq.push(['_setAccount', 'UA-XXXXX-X']);
_gaq.push(['_setLocalRemoteServerMode']);
_gaq.push(['_trackPageview']);

So all on would need to do is add a line of code with a call to _setLocalRemoteServerMode to the Google Analytics Tracking code.

That’s a snap!

Server Log Files

The data should be recorded on your server, whether Apache, Microsoft or otherwise in the server log file.

Brian provides the example in Advanced Web Metrics for an Apache server of:

79.79.125.174 advanced-web-metrics.com - [03/Jan/2010:00:17:01 +0000]
 "GET /images/book-cover.jpg HTTP/1.1" 200 27095
"http://www.advanced-web-metrics.com/blog/2008/02/16/accuracy-whitepaper/"
"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.9.15)
Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.530729"
"__utma=202414657.217961957.1257207415.1257207415.1257207415.1;
__utmb=202414657.1.10.1257207415;__utmc=202414657;
__utmz=2024146571257207415.1.1.utmcsr=google.co.uk|
utmccn=(referral)|utmcmd=referral|utmcct=/imgres;
session_start_time=1257207419839"

Log files are often trimmed by hosting companies after a week, or if they are over a certain size, so make sure to set up a Cron job to either move the data to another part of the server or process it outright.

Bash scripts to accomplish this task abound on the internet.

Data Processing

Each record is a single line in the server log, and as you already know they can get rather large. Importing the data into Excel is not an option.

Individual needs combined the numerous options available make it nearly impossible to cover the universe of possibilities, consider an example where you want the date, time and page visited.

Depending on the flavor of RegEx your programming language uses your use case may vary, but this should grab the data from the server log file:

.*\[([^:]+:[0-9]+:[0-9]+:[0-9]+).*\"(http.*/)".\".*\"

Charting The Visits

I will post some code on how to export the data into an Excel compatible file later in the week, check back for that and some other illustrations with server log data.

Need help with your Google Analytics?

I am always looking to connect with hard working, smart people who have interesting projects.

Send me an email or give me a holler on Twitter.

Technorati Tags: Analytics, Google Analytics, Measure

Share
This entry was posted in Google Analytics and tagged , , . Bookmark the permalink.

2 Responses to Keep a Local Copy of Google Analytics With More Precise Timestamp #measure

  1. Pingback: Tweets that mention Keep a Local Copy of Google Analytics With More Precise Timestamp #measure | MichaelDHealy.com -- Topsy.com

  2. Tom Metro says:

    “…store a copy of your Google Analytics locally on your server.”

    It looks like in your example you are capturing ordinary server logs that include cookies, which happen to have some GA fields.

    I’m not sure that counts as “a copy of your Google Analytics.”

    And then later you are suggesting processing the log so it can be loaded into Excel. Huh? Do we have a wealth of tools designed specifically for processing common log format server logs? Are you reinventing the wheel, or am I not understanding what you are trying to do?

    -Tom

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>