Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Do Linux based spiders and crawlers count? If so, has that skewed the results in a meaningful way?


Unless the web crawlers are intentionally spoofing their user agent to identify as browsers, I don't think that they are counted.


Whether they count or not, I would expect to see "regular" users to be orders of magnitude more common than web crawlers.


But web crawlers generate orders of magnitude more traffic than regular users.


True, but if you're tracking users, you'll normalize for that (tracking cookies, bot exclusion, IP, etc.). It's somewhat nontrivial, but most web analytics packages will give you a decent first cut.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: