Understanding Server Usage Statistics

Fake Visits Don't Help Promote My Book

Before tackling the technical side of website monitoring and maintenance, it makes sense to recall the goal of your website. If the goal was to land a publishing contract and you've gotten an offer thanks to a desperate acquisitions editor finding you through your website, it doesn't really matter whether you are drawing a hundred visitors a day or a hundred thousand. Neither are likely scenarios, the cold calling editor with the contract offer or the hundred thousand visitors a day, but the message is that you need keep your eyes on the ball. If you've already published some books or ebooks, your main goal for the website is probably selling them. The main job of my website is to promote my books. If you're only getting a hundred visitors a day, you're going to have to watch the sales results over an extended period of time, probably months, to get any statistically significant results. I used to waste a lot of time tweaking my order pages based on a couple days sales, but most of the gains or losses proved to be illusory, despite the fact I was always selling a couple dozen books a week. There are all sorts of outside issues, such as holidays, seasonality, national news and weather events, that can impact your sales over the short run. So while that bottom line remains the bottom line, it's important to view your sales results in the context of the overall performance of the website.

Direct reader feedback is easy to understand when the correspondents are sane, which isn't always a given. If somebody writes to say they love a certain page on your site but they can't find any information about a particular subject, and that certain page is all about that particular subject, the usual reason is they are too lazy to read. Every week I get an e-mail or two from people saying, "I've read everything on your site and it's all terrific, but I can't find an answer to this one question..." I'm usually too nice to point them to the page where the information appears, I just type out the answer, but I know they didn't make a big effort. As soon as some visitors see a contact option, they stop reading and they just send you their questions. Don't take it as a criticism of your web design.

But most of my reader feedback comes from people who have read substantial chunks of my website, and when they ask why I haven't covered a particular facet of a subject, or suggest that I expand in a new direction, I take it seriously. In fifteen years of web publishing I've never gotten a technical support question about a website because I'm not loading any fancy graphics or demanding that visitors have Flash installed or javascript enabled. I do get several e-mails a year pointing out typos; sometimes I fix them, sometimes I don't. The way I look at it, you're getting to read the content for free so don't complain about my grammar. If you buy one of my books, then you get the editing. I enjoy getting blessings from religious people, I could live without the curses from the nut jobs, but overall, I'd say that the constant stream of feedback is almost as satisfying as earning a living.

Another type of feedback I can live without start with "You're great and I've linked you" and go on to say how they expect a link in return, blah, blah, blah. I never respond to these e-mails or provide links. I do occasionally look at the websites people are pushing, and they are usually spam factories with no content. I also get a lot of emails from people asking if they can use some of my content for their website, homework, book or good cause. I never give permission for people to repost my pages on their own site - with duplicate content penalties, it can't help and may hurt. I also encourage students to do their own homework, but I'm flexible about books and some charitable usages and recently gave the Peace Corpsthe right to reuse a substantial amount of my work. It's all a good indication that you're doing something right if people want to reproduce your content.

Learning from your website can go way beyond the results and visitor feedback that are intuitively obvious. All commercial websites should come with a free statistics package that tracks the arrival of visitors to your website, along with quite a bit of information about where they are coming from, why they are coming and what they do when they get there. If you don't have a server usage statistics package, or if you want to double up for a better understanding of what's going on as I do, I recommend Google Analytics (also free). Almost as valuable as the information itself is the progression, the fact that it's tracked over time. For example, let's look at two pages I added this year, both of which are successful by my standards, but which followed different growth trajectories.

Gradual Vistor Growth Over Time

The graph above shows the total visitor traffic to a new page about home finance that I added to my website at the beginning of April. I don't have a whole lot of writing about finance on my website, but I was able to link the page from some blog posts I wrote about house hunting, and to some older articles about investing. The initial visitors to the page, a handful one day, none the next, were through those internal links. Then, the search engines, and in particular Google, found the site and the traffic began to average fifteen or twenty-five visitors a day. That initial traffic was enough for the page to pick up some external links, and the traffic has risen steadily ever since. It's also somewhat typical for a summer traffic pattern, dipping a little at the end of August and then starting a rise into September. The sudden drop at the end is just an artifact of my drawing the graph first thing in the morning. Currently, about 80% of the visitors come directly from search engines, while only 5% are from internal or external link referrals. The remaining 15% are reported as "Direct traffic" which is a combination of bots (automated programs running around the web), spiders (search engine indexers) and real people who bookmarked the page and are returning.

This second graph shows visitors to a page which grew very quickly from the moment I put it online, jumped again when the search engines found it, and then stabilized just as quickly. Currently, about 44% of the visitors come directly from search engines, while 40% are from internal or external link referrals. The remaining 16% are reported as "Direct traffic" which is a combination of bots, spiders, and real people who bookmarked the page and are returning. The reason for the fast launch is the high proportion of visitors getting to the page through links, in most cases, internal navigation links. The reason there are so many people arriving through internal navigation links is that this page was a perfect match for some related material that had been online for years and was already drawing strong search engine traffic. A single page on the website, which serves as a sort of table of contents for the related material, is alone responsible for directing about 15% of the visitors to this page.

For what it's worth, the keywords related to home finance for which the search engines direct visitors to the first example are far more popular than the keywords used to reach the second example, a niche technical subject. The first example is a page that is winning a very small slice of a very large pie, while the second example is winning a relatively large slice of a much, much smaller pie. So the examples help illustrate both sides of the niche content argument. You can compete with the biggest web sites and get visitors for your writing on mainstream subjects, you just won't get a large percentage of the visitors who are searching. Fortunately, the numbers of searchers are so very large, that an infinitesimal percentage of them may add up to a hundred visitors a day to a single page. On the other hand, if you have an expertise in a very niche subject, your website can come to dominate that niche overtime, and individual pages will receive a sizable percentage of the Internet search traffic on your subject.

Unfortunately, the beautiful and detailed statistics about visitors contain what engineers call, "a lot of noise". When this noise is transient in nature, such as a one day spike in visitors from a particular site, it's easy enough to visit that site and find out if traffic is real or fake. Some website owners, in an attempt to convince advertisers that their website gets a lot of traffic, will go to any length to get their numbers up. One method is to send fake visits to thousands of other websites every day, in hopes that some webmasters will notice them and check out the site which is apparently sending them. As long as the number of such visits to your website is relatively low, it doesn't hurt you in any way, and after a while, you'll learn to recognize the domain names in your statistics package that are very unlikely to be sending you real visitors. If you sign up for Google's free Analytics program, it will eliminate most kinds of noise, but not all.

As your site matures, you may also see a lot of fake visitors from random IP addresses that that look like direct traffic to a server based statistics package. If that number grows slowly, it may be impossible to differentiate the real human visitors from bot nets (slave computers that have been taken over by intrusive programs and do the bidding of their remote master) or from programs that are designed to generate fake traffic. In either case, the main indication is a high percentage of direct traffic to a particular page that has no reason to be attracting repeat visitors. Normally, direct traffic is generated by people who have bookmarked your pages and are returning to it, who have typed your URL into the their browser, or clicked on a link in a blog subscribed to through certain e-mail clients . It's normal for pages like blogs, or websites like newspapers and banks, to have a high percentage of repeat visitors. People bookmark those sites and return on a regular basis. It's not normal for your static webpage with an article or a short essay to have a high percentage of direct traffic. Why would anybody return to read the same page over and over again? One way to establish a baseline for a page that seems to be getting an unhealthy amount of direct traffic is to compare the results to similar pages on your website.

An simple way to promote your book is to establish yourself in a public forum simply by answering questions or sharing your experiences, and to start meeting other writers and professionals. It takes an investment of time and trust. In my active list participation days, simply keeping up with the posts and side chatter in a professional author's forum cost me over an hour a day. The hidden cost was counted in procrastination. It's easy to fall into a pattern of replacing productive work with professional correspondence that feels productive, or at least holds out the promise that it has some intrinsic reputation value. If you have an income and a lot of time on your hands, I suppose there are worse ways to spend your time, but being an online chatterbox isn't a living.

I have to admit a similar problem with studying website statistics. Spending a ten minutes a week once you know what to look at is more than enough to monitor the progress of your site and pick up on any big events, good or bad. I probably spend fifteen minutes a day, sometimes longer, primarily looking at the new links coming into my site or trying some of the key phrases I see in search engines to see how I'm ranking. Based on the results, I'll occasionally waste a whole day (over the course of a week) on SEO, Search Engine Optimization. I'll try to tweak a page to do just a little better for a give search phrase by changing the internal navigation links on my site, editing the text with that phrase in mind, or trying to find one or two new relevant sites to link me. The former two are probably a complete waste of time these days and may even be counterproductive if the search engine assesses an SEO penalty. Getting new links is always useful, even if it doesn't help with the search presence for that phrase. SEO can be almost as addictive as blogging, so you're better off trying not to fine tune your results as long as your already reaching the audience you'd hoped to attract.

One of the newer options for building your profile on the web that does attract me is creating author videos, due to Google's recent integration of YouTube results with their basic search, . I'm taking about lecture type material here, or how-to, rather than dramatic readings. It's a very different use for video than all of the budding entertainers who sing their song or dance their dance, and from my perspective, the main use to the author is to get a link to their website in there. But, I can see it turning into a huge waste of time with some writers, who are often aspiring script writers in any case. We all have a limited amount of time in the day, so if making it as a writer is a thing, build the traffic to your writing website until you can either see that it's working or that it's not going to work.