1.
Spiders/Crawlers/Scrapers:
2.
3080 msnbot/2.0b (+http://search.msn.com/msnbot.htm)._
3.
105 Yandex/1.01.001 (compatible; Win16; I)
4.
96 Baiduspider+(+http://www.baidu.com/search/spider.htm)
5.
35 msnbot/2.0b (+http://search.msn.com/msnbot.htm)
6.
14 Voyager/1.0
7.
12 KIT webcrawler/0.2.4
8.
8 Baiduspider+(+http://www.baidu.jp/spider/)
9.
5 bitlybot
10.
5 Virtual Reach Newsclip Collector
11.
4 igdeSpyder (compatible; igde.ru; +http://igde.ru/doc/tech.html)
12.
4 facebookexternalhit/1.0 (+https://www.facebook.com/externalhit_uatext.php)
13.
3 Sosospider+(+http://help.soso.com/webspider.htm)
14.
3 ia_archiver (+http://www.alexa.com/site/help/webmasters;
[email protected])
15.
3 Feed43 Proxy/1.0 (
www.feed43.com)
16.
2 Pathtraq/0.1 (+http://pathtraq.com/)
17.
2 Pathtraq/0.1 Gungho/0.09008
18.
1 CyberPatrol SiteCat Webbot (
www.cyberpatrol.com/cyberpatrolcrawler.asp)
19.
1 Begun Robot Crawler
20.
1 CFNetwork/129.24
21.
1 DoCoMo/2.0 N902iS(c100;TB;W24H12)
22.
1 DoCoMo/2.0 F01A(c100;TB;W24H17)
23.
1 Twingly Recon
24.
25.
26.
Malicious software:
27.
1 OSSProxy 1.3.325.351 (Build 325.351 Win32 en-us)(Dec 21 2009 17:36:40)
28.
29.
30.
possible scrapers:
31.
872 libwww-perl/5.832
32.
545 curl/7.18.2 (x86_64-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.8 libssh2/0.18
33.
69 SWF Studio
34.
54 HacketyHack/3.1415927
35.
32 Wget/1.11.4
36.
6 PEAR HTTP_Request class (
http://pear.php.net/ )
37.
5 JNLP/6.0 javaws/1.6.0_14 (b08) Java/1.6.0_14
38.
3 Java/1.6.0_17
39.
3 Java/1.6.0_15
40.
2 Wget/1.12 (linux-gnu)
41.
2 lwp-trivial/1.41
42.
2 curl/7.18.0 (x86_64-pc-linux-gnu) libcurl/7.18.0 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.1
43.
1 libwww-perl/5.813
44.
1 libwww-perl/5.801
45.
1 Wget/1.10.2 (Red Hat modified)
46.
1 Java/1.6.0_18
47.
1 Java/1.6.0_13
48.
1 VB Project
49.
1 Twisted PageGetter
50.
1 PycURL/7.18.2
51.
1 IrssiUrlLog/0.2
52.
1 Download Ninja 7.0
53.
54.
55.
Bittorrent clients:
56.
454 BitTorrent/6400(18095)
57.
81 uTorrent/2000(18488)
58.
69 BTWebClient/6400(18095)
59.
21 BitTorrent/3.4.2
60.
11 uTorrent/1830(16010)
61.
8 BTWebClient/2000(18620)
62.
63.
64.
WordPress sites:
65.
131 WordPress/2.9.2;
http://randomc.dreamhosters.com 66.
90 WordPress/2.3.3
67.
12 WordPress/2.9.2;
http://tigr.ic.cz 68.
6 WordPress.com mShots;
http://support.wordpress.com/contact/ 69.
3 WordPress/2.9.2;
http://anm.zones.lt 70.
1 WordPress/8758;
http://cinetopia.ws 71.
1 WordPress/2.9.2;
http://cubed.thoughts/wp 72.
73.
74.
weird shit:
75.
399 Mediapartners-Google
76.
218 1
77.
59 Internet Explorer
78.
43 Lacave.net RSS2NNTP aggregator v0.6;
[email protected] 79.
40 Microsoft Windows Network Diagnostics
80.
33 100pulse/1.0
81.
29 Yahoo Pipes 1.0
82.
13 Mozilla/4.0 (vBSEO)
83.
10 Mozilla/5.0 (Sage)
84.
9
www.bestpersons.ru/;
[email protected] 85.
4 /dev/null
86.
2 Drupal (+http://drupal.org/)
87.
1 My-UA-String
88.
1 Mozilla
89.
1 User-Agent
90.
1 UrlSearchHook
91.
92.
93.
================================================================================
94.
95.
96.
Mobile clients:
97.
15 HTC_HD2_T8585 Opera/9.7 (Windows NT 5.1; U; en)
98.
9 Nokia7610/2.0 (5.0509.0) SymbianOS/7.0s Series60/2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0
99.
4 BlackBerry9000/4.6.0.304 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/102
100.
4 BlackBerry8530/5.0.0.395 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/104
101.
4 Nokia2600c-2/2.0 (07.60) Profile/MIDP-2.1 Configuration/CLDC-1.1
102.
2 BlackBerry9700/5.0.0.545 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/124
103.
2 BlackBerry8330/4.3.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/105
104.
2 SonyEricssonW595a/R3DA Browser/NetFront/3.4 Profile/MIDP-2.1 Configuration/CLDC-1.1 JavaPlatform/JP-8.3.2
105.
1 UCWEB7.0.1.34/28/999
106.
107.
108.
Console clients:
109.
73 Mozilla/5.0 (PLAYSTATION 3; 1.00)
110.
18 Opera/9.30 (Nintendo Wii; U; ; 3642; en)
111.
7 Opera/9.30 (Nintendo Wii; U; ; 2047-7; en)
112.
5 Mozilla/4.0 (PSP (PlayStation Portable); 2.00)
113.
3 XBMC/9.11 r26017 (Windows; Windows build 7600; SVN r26017;
www.xbmc.org)
114.
115.
116.
Other clients:
117.
19 Microsoft Office/14.0 (Windows NT 6.1; Microsoft Outlook 14.0.4734; Pro; ms-office; MSOffice 14)
118.
8 Microsoft Office/14.0 (Windows NT 6.1; Microsoft Outlook 14.0.4536; Pro; MSOffice 14)
119.
1 Outlook-Express/7.0 (MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; Tablet PC 2.0; TmstmpExt)
120.
1 iTunes/9.1 (Windows; Microsoft Windows XP Professional Service Pack 3 (Build 2600)) AppleWebKit/531.22.7
121.
1 iTunes/9.1 (Windows; Microsoft Windows Vista Home Premium Edition Service Pack 1 (Build 6001)) AppleWebKit/531.22.7
122.
1 iTunes/9.0.3 (Windows; Microsoft Windows XP Home Edition Service Pack 3 (Build 2600)) AppleWebKit/531.21.8
123.
1 iTunes/9.0.3 (Windows; Microsoft Windows Vista x64 x64 Home Basic Edition Service Pack 1 (Build 6001)) AppleWebKit/531.21.8
124.
125.
126.
RSS readers:
127.
666 Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 2 subscribers; feed-id=.........
128.
168 Feedreader 3.14 (Powered by Newsbrain)
129.
129 Wizz RSS News Reader
130.
122 Windows-RSS-Platform/2.0 (MSIE 8.0; Windows NT 6.0)
131.
111 RSSOwl/2.0.4.201002242052 (X11; U; en)
132.
106 Akregator/1.2.9; librss/remnants
133.
102 rss2email/2.65 +http://www.aaronsw.com/2002/rss2email/
134.
96 Apple-PubSub/65.12.1
135.
91 Akregator/1.6.2; syndication
136.
86 Akregator/1.6.1; syndication
137.
84 SharpReader/0.9.7.0 (.NET CLR 1.1.4322.2032; WinNT 5.1.2600.0)
138.
77 RssBandit/1.9.0.972 (.NET CLR 2.0.50727.4927; WinNT 6.1.7600.0;
www.rssbandit.org)
139.
75 RSSOwl/2.0.4.201002242040 (Windows; U; en)
140.
61 Apple-PubSub/65.11
141.
55 Windows-RSS-Platform/2.0 (MSIE 8.0; Windows NT 6.1)
142.
53 Windows-RSS-Platform/2.0 (MSIE 8.0; Windows NT 5.1)
143.
49 Bloglines/3.1 (
www.bloglines.com; 1 subscriber)
144.
37 rss2email/2.61-knux1.41 +http://www.aaronsw.com/2002/rss2email/
145.
36 Liferea/1.4.28 (Linux; en_GB.utf8;
http://liferea.sf.net/)
146.
25 NetNewsWire/3.2.3 (Mac OS X;
www.newsgator.com/Individuals/NetNewsWire/)
147.
23 Akregator/1.5.4; syndication
148.
22 Mozilla/5.0 NewsFox/1.0.5
149.
20 Liferea/1.0.27 (Linux; en_US.UTF-8;
http://liferea.sf.net/)
150.
19 Liferea/1.6.0 (Linux; ru_RU.UTF-8;
http://liferea.sf.net/)
151.
18 Liferea/1.6.2 (Linux; en_US.UTF-8;
http://liferea.sf.net/)
152.
15 FeedDemon/2.6 (
www.newsgator.com/; Microsoft Windows XP)
153.
13 Apple-PubSub/65
154.
12 FeedBurner/1.0 (
www.FeedBurner.com)
155.
11 Netvibes (
www.netvibes.com/; feedId: 7471347)
156.
11 JetBrains Omea Reader 2.2 (
www.jetbrains.com/omea/reader/)
157.
10 Canto/0.7.5 +
http://codezen.org/canto 158.
8 RssBandit/1.8.0.870 (.NET CLR 2.0.50727.3603; WinNT 5.1.2600 Dodatek Service Pack 3;
www.rssbandit.org)
159.
8 FeedDemon/2.7 (
www.newsgator.com/; Microsoft Windows)
160.
7 UniversalFeedParser/4.1 +http://feedparser.org/
161.
6 Snarfer/1.0.2 (
www.snarfware.com/)
162.
4 Liferea/1.7.4 (Linux; en_CA.UTF-8;
http://liferea.sf.net/)
163.
4 SharpReader/0.9.7.0 (.NET CLR 2.0.50727.4927; WinNT 6.1.7600.0)
164.
4 RSSOwl/2.0.4.201002242052 (Windows; U; en)
165.
3 Windows-RSS-Platform/1.0 (MSIE 7.0; Windows NT 6.0)
166.
2 NetNewsWire/3.1.5 (Mac OS X;
www.newsgator.com/Individuals/NetNewsWire/)
167.
1 rss2email/2.62 +http://www.aaronsw.com/2002/rss2email/
168.
169.
170.
================================================================================
171.
172.
173.
uncategorized shit:
174.
73 Citadel 7.61
175.
36 Lynx/2.8.7dev.9 libwww-FM/2.14 SSL-MM/1.4.1
176.
17 Vienna/2.5.0.2501
177.
8 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;)
178.
7 Lynx/2.8.3rel.1 libwww-FM/2.14 OpenSSL/0.9.6
179.
7
http://Anonymouse.org/ (Unix)
180.
6 W3C_Validator/1.767
181.
6 Netvibes Wasabi-bot v1.0
182.
5 PostRank/2.0 (postrank.com; 1 subscribers)
183.
3 Vienna/2.3.2.2303
184.
3 centerim/4.22.9
185.
2 Lynx/2.8.6rel.5 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.8m
186.
2 gnome-vfs/2.16.3 neon/0.25.4
187.
1 Safari (unknown version) CFNetwork/454.10.4
188.
1 SafariBookmarkChecker/1.30 (+http://www.coriolis.ch/)
189.
1 PostRank/2.0 (postrank.com)
190.
1 ovale_total/0.1
191.
1 JS-Kit URL Resolver,
http://js-kit.com/
