-
Notifications
You must be signed in to change notification settings - Fork 1
/
crawler.txt
1075 lines (1075 loc) · 120 KB
/
crawler.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Pinterest/0.2 (+http://www.pinterest.com/)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google Favicon
Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)
Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)
Mozilla/5.0 (compatible; Pinterestbot/1.0; +http://www.pinterest.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Mediapartners-Google
Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)
Pinterest/0.2 (+http://www.pinterest.com/bot.html)
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)
Twitterbot/1.0
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google Favicon
Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0
ADmantX Platform Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)
Google favicon
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b
Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)
admantx-usastn/2.4 (+http://www.admantx.com/service-fetcher.html)
ADmantX Platform Sync US Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com
ADmantX Platform Semantic Analyzer US - Turn - ADmantX Inc. - www.admantx.com - support@admantx.com
Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36 evaliant
Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot)
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1 (KHTML, like Gecko) CriOS/51.0.2704.104 Mobile/13F69 Safari/601.1.46 evaliant
ADmantX Platform Semantic Analyzer Appnexus - ADmantX Inc. - www.admantx.com - support@admantx.com
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )
admantx-sap/2.4 (+http://www.admantx.com/service-fetcher.html)
msnbot/2.0b (+http://search.msn.com/msnbot.htm)
Sosospider+(+http://help.soso.com/webspider.htm)
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
facebookexternalhit/1.1;line-poker/1.0
ADmantX Platform Semantic Analyzer - ADform - ADmantX Inc. - www.admantx.com - support@admantx.com
admantx-adform/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36 evaliant
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Pinterestbot/1.0; +http://www.pinterest.com/bot.html)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36 Squider/0.01
Mozilla/5.0 (Linux; Android 5.0.2; LG-V410/V41020c Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/34.0.1847.118 Safari/537.36 evaliant
admantx-euastn/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit (KHTML, like Gecko) Mobile (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
admantx-adform/2.5 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (Linux; Android 4.0.3; HTC One X Build/IML74K) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19 evaliant
Mozilla/5.0 (linux; u; android 4.1.1; ja-jp; sgpt12 build/tjs0166) applewebkit/534.30 (khtml, like gecko) version/4.0 safari/534.30 yjapp-android jp.co.yahoo.android.yjtop/2.1.8 evaliant
Mozilla/5.0 (iPad; CPU OS 7_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A406 Safari/8536.25 evaliant
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.4.2; id; SM-G900 Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/9.9.2.467 U3/0.8.0 Mobile Safari/534.30 evaliant
Mozilla/5.0 (Linux; Android 6.1.1; SAMSUNG SM-G925F Build/LMY47X) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/4.0 Chrome/51.0.2704.81 Mobile Safari/537.36 evaliant
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13F69 [FBAN/FBIOS;FBAV/59.0.0.51.142;FBBV/33266808;FBRV/0;FBDV/iPhone7,1;FBMD/iPhone;FBSN/iPhone OS;FBSV/9.3.2;FBSS/3;FBCR/Telkomsel;FBID/phone;FBLC/en_US;FBOP/5] evaliant
Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; )
admantx-adform/3.0 (+http://www.admantx.com/service-fetcher.html)
ADmantX Platform Semantic Analyzer - Turn EU - ADmantX Inc. - www.admantx.com - support@admantx.com
Mozilla/5.0 (Linux; U; Android 6.0.1; en-US; SM-G920F Build/LMY47X) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/10.10.0.796 U3/0.8.0 Mobile Safari/534.30 evaliant
Mozilla/5.0 (Linux; U; Android 5.0.1; en-US; GT-I9505 Build/LRX22C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/10.9.8.770 U3/0.8.0 Mobile Safari/534.30 evaliant
Mozilla/5.0 (iPhone; CPU iPhone OS 8_1_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) CriOS/47.0.2526.70 Mobile/12B436 Safari/600.1.4 (000410) evaliant
Mozilla/5.0 (compatible; MJ12bot/v1.4.3; http://www.majestic12.co.uk/bot.php?+)
Mozilla/5.0 (Linux; Android 5.0.2; SAMSUNG SM-T550 Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/3.3 Chrome/38.0.2125.102 Safari/537.36 evaliant
Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG SM-T715Y Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/4.0 Chrome/44.0.2403.133 Safari/537.36 evaliant
Mozilla/5.0(compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)
Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)
ADmantX Platform Semantic Analyzer - APAC - ADmantX Inc. - www.admantx.com - support@admantx.com
facebookexternalhit/1.1
ExB Language Crawler 2.1.5 (+http://www.exb.de/crawler)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 evaliant
Mozilla/5.0 (linux; u; android 4.1.1; ja-jp; sgpt12 build/tjs0166) applewebkit/534.30 (khtml, like gecko) version/4.0 safari/534.30 yjapp-android jp.co.yahoo.android.yjtop/2.1.8
Mozilla/5.0 (iPad; CPU OS 9_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) GSA/8.0.57838 Mobile/13B5110e Safari/600.1.4 evaliant
Mozilla/5.0 (compatible; Yahoo Link Preview; https://help.yahoo.com/kb/mail/yahoo-link-preview-SLN23615.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (iPad; CPU OS 8_0 like Mac OS X) AppleWebKit/537.51.3 (KHTML, like Gecko) Version/8.0 Mobile/11A4132 Safari/9537.145 evaliant
Mozilla/5.0 (iPad; CPU OS 9_3_3 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13G34 Safari/601.1 evaliant
admantx-apacas/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Chrome/27.0.1453 Safari/537.36 GoogleBot/2.1
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 GoogleBot/2.1
Mozilla/5.0 (compatible; JikeSpider; +http://shoulu.jike.com/spider.html)
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A evaliant
Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com)
Mozilla/5.0 (compatible; AhrefsBot/5.1; +http://ahrefs.com/robot/)
Mozilla/5.0 (compatible; CareerBot/1.1; +http://www.career-x.de/bot.html)
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36 QIHU 360SE; 360Spider
admantx-euastn/2.5 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (compatible; news bot /2.1)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html)
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDR; .NET4.0C; .NET4.0E; .NET CLR 1.1.4322; Tablet PC 2.0); 360Spider
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27 evaliant
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)
admantx-apacas/2.5 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/27.0.1453 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible;acapbot/0.1;treat like Googlebot)
Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)
Slackbot-LinkExpanding (+https://api.slack.com/robots)
msnbot/2.0b (+http://search.msn.com/msnbot.htm)._
Yahoo! Slurp China
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b
panscient.com
Mozilla/5.0 (compatible; AhrefsBot/4.0; +http://ahrefs.com/robot/)
Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://import.io)
UnwindFetchor/1.0 (+http://www.gnip.com/)
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) NOT Firefox/3.5
admantx-usasab/3.0 (+http://www.admantx.com/service-fetcher.html)
TelegramBot (like TwitterBot)
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider
Mozilla/4.0 (MSIE 6.0; Windows NT 5.1; Search)
Mozilla/5.0 (compatible; MJ12bot/v1.4.6; http://mj12bot.com/)
bitlybot
Googlebot
Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); +http://www.exabot.com/go/robot)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/)
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.2.2661.102 Safari/537.36; 360Spider
Mozilla/5.0 (compatible; Butterfly/1.0; +http://labs.topsy.com/butterfly/) Gecko/2009032608 Firefox/3.0.8
Googlebot-Image/1.0
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)
Mozilla/5.0 (compatible; Yahoo Ad monitoring; https://help.yahoo.com/kb/yahoo-ad-monitoring-SLN24857.html)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36
Nutraspace/Nutch-1.2 (www.nutraspace.com)
Mozilla/4.0 (Add Catalog; Windows NT 5.1; infobot.4942@yahoo.com)
admantx-ussy02/2.4 (+http://www.admantx.com/service-fetcher.html)
facebookexternalhit/1.1;kakaotalk-scrap/1.0; +https://devtalk.kakao.com/t/scrap/33984
Mozilla/5.0 (compatible; TweetmemeBot/3.0; +http://tweetmeme.com/)
ia_archiver
msnbot-NewsBlogs/2.0b (+http://search.msn.com/msnbot.htm)
YahooCacheSystem
Mozilla/5.0 (compatible; AhrefsBot/5.2; News; +http://ahrefs.com/robot/)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like Gecko; Google Page Speed Insights) Chrome/22.0.1229 Safari/537.4 GoogleBot/2.1
Pinterest/0.1 +http://pinterest.com/
Mozilla/5.0 (Admin Catalog; Windows NT 5.1; infobot.4942@yahoo.com)
Mozilla/5.0 (compatible; Discordbot/1.0; +https://discordapp.com)
CatchBot/2.0; +http://www.catchbot.com
Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot)
bitlybot/3.0 (+http://bit.ly/)
Mozilla/5.0 (compatible; MJ12bot/v1.4.2; http://www.majestic12.co.uk/bot.php?+)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview Analytics) Chrome/41.0.2272.118 Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
OnPageBot (compatible; Googlebot 2.1; +https://bot.onpage.org/)
facebookexternalhit/1.1;kakaotalk-scrap/1.0;
Google Crawler: Googlebot/2.1 (+http://www.google.com/bot.html)
ExB Language Crawler 2.1.3 (+http://www.exb.de/crawler)
Mozilla/5.0 (compatible; AhrefsBot/5.2; SA; +http://ahrefs.com/robot/)
Y!J-ASR/0.1 crawler (http://www.yahoo-help.jp/app/answers/detail/p/595/a_id/42716/)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/[WEBKIT_VERSION] (KHTML, like Gecko, Mediapartners-Google) Chrome/[CHROME_VERSION] Safari/[WEBKIT_VERSION]
Googlebot/2.1; +http://www.google.com/bot.html)
admantx-ussy02/2.5 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (Linux; Android 5.1; Nexus 7 Build/LMY47D) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/39.0.0.0 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.0.26
Mozilla/5.0 (YahooYSMcm/3.0.0; http://help.yahoo.com)
Mozilla/5.0 (compatible; Googlebot/2.1 +http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; special_archiver/3.1.1 +http://www.archive.org/details/archive.org_bot)
Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; no-reply@alexa.com)
Mozilla/4.0 (compatible; Netcraft Web Server Survey)
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)
Yahoo Crawler: Mozilla/5.0 (compatible; Yahoo! Slurp;http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)
Mozilla/5.0 (compatible;acapbot/0.1.;treat like Googlebot)
Mozilla/5.0 (Linux; Android 6.0.1; SC-02G Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/56.0.2924.87 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.6
Content Crawler Spider
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; TweetmemeBot/2.11; +http://tweetmeme.com/)
YahooSeeker-Testing/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; Yahoo! Search - Web Search)
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11; 360Spider
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (compatible; archive.org_bot; Archive-It; +http://archive-it.org/files/site-owners.html)
Mozilla/5.0 (compatible; Yahoo Pipes 2.0; +http://developer.yahoo.com/yql/provider) Gecko/20090729 Firefox/3.5.2
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html
Mozilla/5.0 (compatible; 008/0.85; http://www.80legs.com/webcrawler.html) Gecko/2008032620
Apple-iPhone4C1/1001.523 (compatible;acapbot/0.1;treat like Googlebot)
ADmantX Platform Semantic Analyzer EU - ADmantX Inc. - www.admantx.com - support@admantx.com
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/Nutch-2.3.1
findlinks/2.0.2 (+http://wortschatz.uni-leipzig.de/findlinks/)
ADmantX Platform Sync US 2 Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)
findlinks/2.6 (+http://wortschatz.uni-leipzig.de/findlinks/)
Mozilla/5.0 (Linux; Android 4.4.2; SH-04F Build/S7160) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.16
Mozilla/3.0 (Content Parser; Windows NT 5.1; infobot.4942@yahoo.com)
Mozilla/5.0(compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
ExB Language Crawler 2.1.4 (+http://www.exb.de/crawler)
Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)
Mozilla/5.0 (compatible; heritrix/3.3.0-SNAPSHOT-20140702-2247 +http://archive.org/details/archive.org_bot)
Mozilla/5.0 (compatible; Exabot/3.0; http://www.exabot.com/go/robot)
Googlebot-News
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; AhrefsBot/3.0; +http://ahrefs.com/robot/)
GoogleBot/3.0
Mozilla/5.0 (compatible; discobot/2.0; +http://discoveryengine.com/discobot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; https://deepcrawl.com/bot)
Mozilla/5.0 (compatible; Googlebot/2.1; https://www.deepcrawl.com/bot)
Googlebot-Video/1.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0+(compatible;+Sosospider/2.0;++http://help.soso.com/webspider.htm)
Mozilla/5.0 (compatible; pub-crawler; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com)
Mozilla/5.0 (compatible; WebHistoryBot/1.2.1 IS NOT SE bot like Googlebot/2.1; +http://www.google.com/bot.html,Yahoo! Slurp or Bingbot)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe src="whatismyip.com"></iframe>
ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3366.99 Safari/537.36 (compatible; Googlebot/2.1; SeoakademiyaBot/1.0; +http://seo-akademiya.com/bot.html)
facebookexternalhit/1.1 (+https://www.facebook.com/externalhit_uatext.php)
Mozilla/5.0 (Linux; U; Android 4.3; zh-tw; HTC Butterfly s Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Googlebot/2.1
Mozilla/5.0 (Linux; Android 4.4.4; 404KC Build/105.0.2900) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.0
Mozilla/5.0 (compatible; OrangeBot/2.0; support.voilabot@orange.com)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html )
Googlebot 2.1
Mozilla/5.0 (Linux; U; Android 4.4.2; zh-tw; HTC_Butterfly Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (compatible; AhrefsBot/3.1; +http://ahrefs.com/robot/)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MTC19Z; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/62.0.3202.84 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.9.0
Mozilla/5.0 (compatible; alexa site audit/1.0; +http://www.alexa.com/help/webmasters; siteaudit@alexa.com)
Mozilla/5.0 (compatible; Twitterbot/1.0)
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/85.0.0.15.70;]
Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1 (GoogleBot)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729; Diffbot/0.1; +http://www.diffbot.com)
Googlebot/2.1; http://www.google.com/bot.html
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://www.majestic12.co.uk/bot.php?+)
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.133 Mobile Safari/537.36
FAST Enterprise Crawler/6.7.0 (crawler@fast.no)
msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.125 Mobile Safari/537.36
Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server;) Daumoa/4.0 (Following Mediapartners-Google)
FAST Enterprise Crawler/5.3.4 (crawler@fast.no)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 5.1; KYV34 Build/104.1.2940; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/42.0.2311.137 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.14
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AlexaToolbar/alxf-2.17
Baiduspider+(+http://www.baidu.com/search/spider.htm)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) FirePHP/0.7.4
Mozilla/5.0 (compatible; bingbot/2.0 http://www.bing.com/bingbot.htm)
NokiaC1-01/2.0 (06.15) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
ia_archiver(OS-Wayback)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/537.4 (KHTML, like Gecko; Google Page Speed Insights) Version/6.0 Mobile/10A525 Safari/8536.25 GoogleBot/2.1
admantx-euaspb/2.5 (+http://www.admantx.com/service-fetcher.html)
Mozilla / 5.0(compatible; bingbot / 2.0; +http://www.bing.com/bingbot.htm)
Googlebot (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
admantx-euaspb/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_5 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/5.4.0
Mozilla/5.0 (iPad; CPU OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13G36 Safari/601.1Mozilla/5.0 (iPhone; CPU iPhone OS 8_1_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) CriOS/47.0.2526.70 Mobile/12B436 Safari/600.1.4 (000410) evaliant
Googlebot/2.1 (+http://www.googlebot.com/bot.html) (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
facebookexternalhit
BotThief (compatible; Googlebot/2.1; Location RU)
Mozilla/5.0+(compatible;+googlebot/2.1;++http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36
bingbot
Mozilla/5.0 (compatible; ia_archiver/1.0; +http://www.alexa.com/help/webmasters; crawler@alexa.com)
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529543865-0
findlinks/2.0.1 (+http://wortschatz.uni-leipzig.de/findlinks/)
Mozilla/5.0 (compatible; Googlebot/2.1; actually this is http://ismyblogworking.com/)
Mozilla/5.0 (+http://www.googlebot.com/bot.html; Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.9200
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) AlexaToolbar/pcPuH3uf-2.0 AlexaToolbar/alxf-2.17
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/3.7.1
http://www.googlebot.com/bot.html
* Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; SCL24 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/51.0.2704.81 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.5.1
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)
admantx-ussy02/3.0 (+http://www.admantx.com/service-fetcher.html)
YahooSeeker/1.2 (compatible; Mozilla 4.0; MSIE 5.5; yahooseeker at yahoo-inc dot com ; http://help.yahoo.com/help/us/shop/merchant/)
Mozilla/5.0 (iPhone; U; CPU iPhone OS) (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529256481-0
ia_archiver/8.2 (Windows 7.6; en-US;)
Mozilla/5.0 (compatible; Googlebot/2.1; http://deepcrawl.co.uk/bot.html)
Mozilla/5.0 (Linux; U; Android 4.0.9; en-us; GT_I9220 Build/Butterfly_V128Cs20120705) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Googlebot/2.1+(+http://www.googlebot.com/bot.html)
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 7.0; STF-AL00; Build/HUAWEISTF-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.92 Mobile Safari/537.36 SogouMSE,SogouMobileBrowser/5.8.18
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +https://www.deepcrawl.com/bot)
findlinks/2.1.5 (+http://wortschatz.uni-leipzig.de/findlinks/)
Sogou News Spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Mozilla/5.0 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm)
facebookexternalhit/1.1;goscraper/1.0;
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.89 Mobile Safari/537.36
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529219557-0
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 6.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
Aensland Mobile Browser /5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
66.249.64.13 - crawl-66-249-64-13.googlebot.com
Mozilla/5.0 (compatible; Googlebot/2.1; +https://whatis.contentkingapp.com/)
Mozilla/5.0 (Linux; Android 6.0.1; HTC Butterfly 2 Build/MMB29M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.116 Mobile Safari/537.36
Baiduspider-image+(+http://www.baidu.com/search/spider.htm)
Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.23
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Baiduspider-image+(+http://www.baidu.com/search/spider.htm)\nReferer: http://image.baidu.com/i?ct=503316480&z=0&tn=baiduimagedetail
AbachoBOT
Mozilla/5.0 (Linux; U; Android 4.2.2; en-us; HTC_Butterfly Build/JDQ39) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (Linux; U; Android 4.3; de-de; HTC_Butterfly Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36.
Mozilla/5.0 (Windows; U; Windows NT 5.1;fr;rv:1.8.1) VoilaBotCollector BETA 0.1 (http://www.voila.com/)
Mozilla/5.0 (compatible; Googlebot/3.2; +http://www.facebook.com)
Googlebot/2.1 Compatible: http://www.googlebot.com/bot.html
Covario-IDS/1.0 (Covario; http://www.covario.com/ids; support at covario dot com)
"GET /robots.txt HTTP/1.1" 404 287 "-" "Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)"
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529256478-0
Mozilla/5.0 (compatible; Windows NT 5.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/535.19 (via Wayback Save Page)
Exabot/2.0
msnbot/1.1 ( http://search.msn.com/msnbot.htm)
Mozilla/5.0 (Linux; Android 5.1; Cosmos V15 Build/LMY47D; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/43.0.2357.121 Mobile Safari/537.36
Googlebot/2.1 ( http://www.googlebot.com/bot.html)
msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Googlebot/2.1" "Compatible"="+http://www.googlebot.com/bot.html
Mozilla/5.0 (Linux; Android 6.0; SO-01H Build/32.1.F.1.38; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.2
Mozilla/5.0 (iPad; CPU OS 9_3_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13E238 YJApp-IOS jp.co.yahoo.ipn.appli/4.3.27
envolk[ITS]spider/1.6 ( http://www.envolk.com/envolkspider.html)
Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy)
Accoona-AI-Agent/1.1.1 (crawler at accoona dot com)
Mozilla/5.0 (compatible;bingbot/2.0;+http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2; ocrawler; omax7828@yahoo.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36
Mozilla/5.0 (iPhone; CPU iPhone OS 11_3_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/5.5.1
cosmos/0.9_(robot@xyleme.com)
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly s Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.59 Mobile Safari/537.36
Mozilla/5.0 (compatible; Baiduspider-cpro; +http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; Googlebot/2.1
Mozilla/5.0 (compatible baidu spider google robot bing sogou yahoo alibaba sm.cn 360)
Googlebot/2.1 (http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; special_archiver/3.3.0 bot@archive.org +https://archive.org/details/archive.org_bot)
Mozilla/5.0 (Linux; Android 7.0; U11_Plus Build/NRD90M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/67.0.3396.87 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.36
Mozilla/5.0 (compatible; Yahoo Ad monitoring)
Mozilla/5.0 (Windows NT 10.0; WOW64) Apple
print(WebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm)
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1; + http://www.google.com/bot.html
Alexabot
Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.12
facebookexternalhit/1.1 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.73.11 (KHTML, like Gecko) Version/7.0.1 Safari/537.73.11
YahooExternalCache
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SHL22 Build/S8202) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.5.1
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.8
Mozilla/5.0 (Linux; Android 5.0.2; HTC_Butterfly_s_901s Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
SurveyBot/2.3 (Whois Source)
Mozilla/5.0 (Linux; Android 5.1.1; SOV32 Build/32.0.D.0.300; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/44.0.2403.117 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.8
Mozilla/5.0 (compatible; Googlebot/wannabe; +https://xfrocks.com/resources/30/)
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
findlinks/2.2 (+http://wortschatz.uni-leipzig.de/findlinks/)
(compatible; Googlebot/2.1; http://www.google.com/bot.html)
Sogou+web+robot+(+http://www.sogou.com/docs/help/webmasters.htm#07)
Mozilla/5.0 (Linux; Android 5.0.2; HTL23 Build/LRX22G; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/56.0.2924.87 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.13
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/82.0.0.23.70;]
Mozilla/5.0 (compatible; Googlebot/2.1; +www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +https://deepcrawl.com/bot)
Mozilla/5.0 (compatible; AhrefsBot/2.0; +http://ahrefs.com/robot/)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0_4 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/1.1.0
boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (compatible; Macintosh; Intel Mac OS X) AppleWebKit/534.46.0 (KHTML, like Gecko) Googlebot/2.1 Safari/7534.34
Googlebot-News (2.3.4.1, ruby 1.9.3 (2014-05-14))
Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; butterfly 3 Build/MocorDroid2.3.5) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0_4 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Mobile/11B554a A2I SogouMobileBrowser/5.3.0
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15F79 webview/d985435f4b3f9793204a2c10a92fbd90 YJApp-IOS jp.co.yahoo.apppkgcal/2.4.6
Y!J-BRJ/YATS crawler (http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html)
Googlebot/2.1 (+http://www.google.com/bot.html
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+) ASProxy/5.5.0
crawl-66-249-66-1.googlebot.com
Mozilla/5.0 ;compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html;
Mozilla/5.0 (compatible; X11; Linux x86_64; Google-StructuredDataTestingTool; +http://www.google.com/webmasters/tools/richsnippets)
Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Append?: Append
MJ12bot/v1.0.5 (http://majestic12.co.uk/bot.php?+)
GoogleBot2/1
Googlebot/Nutch-1.7
Mozilla/5.0 (iPhone; CPU iPhone OS 11_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B93 SogouMobileBrowser/5.10.2
admantx-eusy01/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://myteltech.blogspot.in/help/us/ysearch/slurp)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SO-04E Build/10.3.1.B.2.48) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.13
findlinks/1.1.6-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)
Mozilla/5.0 (compatible; Googlebot/20.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; KYY21 Build/209.0.1700) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.9
Mozilla/5.0 (Linux; Android 5.0.2; F-04G Build/V15R38A; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/44.0.2403.117 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.25
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.78 Safari/537.36 Googlebot-News
New-Sogou-Spider/1.0 (compatible; MSIE 5.5; Windows 98)
Mozilla/5.0 (Linux; Android 5.1.1; Nexus 5 Build/LMY48B; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/44.0.2403.117 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.21
msnbot/1.0 ( http://search.msn.com/msnbot.htm)
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60 YJApp-IOS jp.co.yahoo.ipn.appli/4.7.2
Mozilla/5.0 (Linux; Android 5.0; SC-02G Build/LRX21T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/45.0.2454.95 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.27
Mozilla/5.0 (compatible; AnyApexBot/1.0; +http://www.anyapex.com/bot.html)
Googlebot/4.0 (Blackberry; ExtremeOS 10_3_1 PPC; rv:12.6) Crocodile/20040327 Gridspider/12.6
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; SOV33 Build/35.0.D.0.414; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/54.0.2840.85 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.0.20
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) https://deepcrawl.com/bot
Nokia2700c-2/2.0 (09.98) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/UCWEB8.2.0.116/69/352 UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 6.0.1; HTC Butterfly 2 Build/MMB29M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Mobile Safari/537.36
Mozilla/5.0 (compatible; Googlebot/2.1;)
Mozilla/4.0 (Googlebot/2.1 (+http://www.googlebot.com/bot.html); MSIE 6.0; )
Mozilla/5.0 (Linux; Android 4.4.2; KYY23 Build/104.0.2820) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.27
Mozilla/5.0 (iPad; CPU iPhone OS 8_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/3.11.0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SH-06E Build/SC040) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.14
Mozilla/5.0 (Linux; Android 4.4.2; 302HW Build/HuaweiP6S-L04) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.32
Mozilla/5.0 (Windows NT 6.3; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 (via Wayback Save Page)
Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; HTC_Butterfly Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36 (Pagebot/0.1; +https://www.page.rest/) facebookexternalhit/1.1 W3C_Validator/1.654
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15G77 YJApp-IOS jp.co.yahoo.ipn.appli/4.11.3
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://www.majestic12.co.uk/bot.php?+)/Nutch-1.13
{ "ua": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36", "browser": { "name": "Safari" }, "engine": { "version": "537.36", "name": "WebKit" }, "os": {}, "device": {}, "cpu": {} }
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2236.0 Cosmos/1.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPad; CPU OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Mobile/14E304 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.23
ExB Language Crawler 2.1.2 (+http://www.exb.de/crawler)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529204392-0
Mozilla/4.0 (compatible; B-l-i-t-z-B-O-T)
Mozilla/5.0 (Linux; Android 2.3.2; Nexus One; U; id-en) Build/FRF9 applewebkit/523.18 presto/2.2.1.40 Version/4.0 mobile safari/523.18 Googlebot/20.1.254.89 + http://www.google.com/bot.html UCBrowser8.4.0.159/69/352 UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 4.4.2; SHL25 Build/SC241) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.9.0
schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; Contact: webcrawl@schibstedsok.no;)
dotbot
Mozilla/5.0 (Linux; Android 5.0.1; 503HW Build/HuaweiALE-L02; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.0.23
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.3; en-my; HTC_Butterfly Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
AppEngine-Google; (+http://code.google.com/appengine; appid: ppeeepp) (via Wayback Save Page)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529211523-0
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.2.2.0 Safari/537.31 Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; SO-01J Build/39.0.B.0.361; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/55.0.2883.91 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.2
Mozilla/11.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
GooglebotNSA
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529212604-0
Mozilla/5.0 (compatible; Googlebot/6.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Android; U; Android; en-id; compatible; Googlebot/10.0) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 5.0.1; Nexus 9 Build/LRX22C) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/37.0.0.0 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.38
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.60 Mobile Safari/537.36
Googlebot/2.X (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36
SurveyBot/2.2
Nokia6303iclassic/5.0 (10.80) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (iPad; CPU OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16B92 YJApp-IOS jp.co.yahoo.ipn.appli/4.13.0
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36 OPR/34.0.2036.25 (via Wayback Save Page)
findlinks/2.0.5 (+http://wortschatz.uni-leipzig.de/findlinks/)
Mozilla/5.0 (Linux; Android 5.0.2; HTC Butterfly s Build/LRX22G; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/51.0.2704.81 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/85.0.0.15.70;]
Mozilla/5.0 (Macintosh; Butterfly/1.0; +http://labs.topsy.com/butterfly/) Gecko/2009032608 Firefox/3.0.8
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529219558-0
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.102 Safari/537.36 Vivaldi/2.0.1309.37 Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 7.0; 603SH Build/S0016; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/66.0.3359.126 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.19.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529220165-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529220166-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529220167-0
Googlebot/2.1 (+http://googlebot.com/bot.html)
Mozilla/5.0 (Android 4.1.2; Mobile; rv:44.0) Gecko/44.0 Firefox/44.0 (via Wayback Save Page)
Mediapartners-Google (via Wayback Save Page)
msnbot/0.11 ( http://search.msn.com/msnbot.htm)
Mozilla/5.0 (Linux; Android 9; MIX 2S Build/PKQ1.180729.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.100 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.29.0
Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0; compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html)
Mozilla/5.0 (iPod; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/3.8.0
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528027464-0
Random string containing AhrefsBot and some other stuff
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528027468-0
Mozilla/5.0 (Linux; Android 6.0.1; SHV34 Build/S7160; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/54.0.2840.68 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.0.20
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528647297-0
Mozilla/5.0 (Linux; Android 4.4.4; d-01G Build/HuaweiMediaPad) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.1750.136 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.24
Mozilla/5.0 (Linux; Android 4.4.4; SHV31 Build/SA301) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.18
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1532150122-0
findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)
Mozilla/5.0 (Linux; Android 4.4.2; SO-03F Build/17.1.1.B.3.195) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.33
Googlebot/2.1 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1530131216-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1534948535-0
Mozilla/5.0 (Linux; Android 4.4.2; F-01F Build/V10R22A) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.18
Mozilla/5.0 (compatible; Googlebot/2.1; https://www.deepcrawl.com/bot) https://deepcrawl.com/bot
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528421719-0
Mozilla/5.0 (Linux; Android 6.0; ZTE BA610T Build/LMY47D; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/50.0.2661.86 Mobile Safari/537.36 SogouMSE,SogouMobileBrowser/5.11.8
googlebot 2,12
Mozilla/5.0 (Linux; U; Android 5.1; en-US; Cosmos V4 Build/LMY47D) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.108 UCBrowser/12.0.0.1088 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; Android 4.2.2; en-us; COSMOS X2 Reloaded Build/JOP40D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (iPad; CPU OS 11_2_1 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Mobile/15C153 YJApp-IOS jp.co.yahoo.ipn.appli/4.9.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36 Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) yahoo.adquality.lwd.desktop/1525684924-0
Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; butterfly 3 Build/MocorDroid2.3.5) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1/4.05d.1002.m75.08.12011
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.5 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.71 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1+(+http://www.googlebot.com/bot.html)"
NokiaX2-00/5.0 (04.25) Profile/MIDP-2.1 Configuration/CLDC-1.1 UCWEB/2.0 (Java; U; MIDP-2.0; id; Googlebot) U2/1.0.0 UCBrowser/9.4.1.377 U2/1.0.0 Mobile UNTRUSTED/1.0
Mozilla/5.0 (compatible; Googlebot/1.0; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1532903550-0
Yahoo-MMCrawler/3.x (mm dash crawler at trd dot overture dot com)
Mozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/)
Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.betaspider.com/)
Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)
msnbot-131-253-38-20.search.msn.com
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36 (via Wayback Save Page)
Mozilla/5.0 (compatible? Googlebot/2.1? + Http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529507253-0
Mozilla/5.0 (Linux; Android 7.0; HUAWEI VNS-L22 Build/HUAWEIVNS-L22; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/62.0.3202.84 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.9.1
SurveyBot/2.3+(Whois+Source)
Mozilla/5.0 AppleWebKit/27.0.1453 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/27.0.1453
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527925193-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1533431326-0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; P-03E Build/JDQ39) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.7
Opera/9.80 (J2ME/MIDP; Opera Mini/4.5.33867/34.1697; U; en) Presto/2.8.119 Version/11.10 (via Wayback Save Page)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529507806-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1533192853-0
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) or (rarely used): Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)aaa
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.155 Safari/537.22 Mozilla/5.0 (iPhone; U; CPU iPhone OS) (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)
Mozilla/4.0 (compatible; MSIE 6.0; Googlebot/2.1; +http://www.google.com/bot.html)
Mediapartners-Google AppEngine-Google; (+http://code.google.com/appengine; appid: s~archerfroxy-hrd)
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SO-04E Build/10.1.1.D.2.31) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.27
Googlebot/2.1; +
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36 (via Wayback Save Page)
Googlebot/2.1(+http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1542353920-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528739412-0
GoogleBot/2.9 (Fedora 3.4; en-US;)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529508176-0
BecomeBot
Mozilla/5.0 (Linux; Android 4.4.2; 302KC Build/103.0.2f30) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.27
Mozilla/5.0 (Linux; U; Android 4.3; zh-tw; HTC_Butterfly Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
GoogleBot/6.6 (OS 10.5 7.0; en-US;)
Mozilla/5.0 (Linux; Android 4.4.4; SH-01G Build/S1160) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.5
Mozilla/5.0 (Linux; Android 5.0; ASUS_Z00AD Build/LRX21V; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/47.0.2526.100 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.9.0
Mozilla/5.0 (Linux; Android 4.2.2; HTC Butterfly Build/JDQ39) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.59 Mobile Safari/537.36
YahooSeeker
Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/)
Mozilla/5.0 (compatiable Googlebot/2.1 +http//www.google.com/bot hrml)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; F-02F Build/V21R66B) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.0.29
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528655981-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529509405-0
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X; en-us) AppleWebKit/534+ (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 MsnBot-Media /1.0b
Mozilla/5.0 (compatible; bingbot/2.0; +https://deepcrawl.com/bot)
Mozilla/5.0 (Linux; Android 5.0.2; HTC_Butterfly_s Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
Mozilla/5.0 (Windows NT 6.1; rv:38.0) Gecko/20100101 Firefox/38.0 (via Wayback Save Page)
GoogleBot/5.5 (Windows 95 5.1; en-US;)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SOL23 Build/14.1.C.3.95) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.26
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528040919-0
Mozilla/5.0 (iPhone;<br /> U; CPU iPhone 0S 3.0<br /> like Mac 0S X; en-us;<br /> compatible;<br /> Googlebot/2.1;<br /> http://<br /> www.google.com/<br /> bot.html;<br /> AppleWebKit/528.18(KHTML,like<br /> Gecko) Version/4.0<br /> Mobile/7A341<br /> Safari/528.16
Googlebot/2.1 (+http://www.googlebot.com/bot.html).
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_5 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/3.11.0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SH-05F Build/S3140) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.8.1
Sosospider+(+help.soso.com/webspider.htm)
Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_6 like Mac OS X) AppleWebKit/604.5.6 (KHTML, like Gecko) Mobile/15D100 YJApp-IOS jp.co.yahoo.ipn.appli/4.8.20
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1541056202-0
Mozilla/5.0 (X11; U; Linux armv7l; en-GB; rv:1.9.2.3pre) Gecko/20100723 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11 360Spider;
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529330562-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527848791-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528129762-0
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15F79 SogouMobileBrowser/5.12.9
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527848808-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528129781-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528219171-0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1532574095-0
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)
Mozilla/23.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible;+Googlebot/2.1;+http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe></iframe>
BebopBot/2.5.1 (compatible; media crawler V1; http://www.apassion4jazz.net/bebopbot.html;)
Googlebot/2.1(Windows 3.1)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) AlexaToolbar/alxf-2.17
msnbot/0.11 (+http://search.msn.com/msnbot.htm)
"Googlebot/2.1",gzip(gfe) (via translate.google.com)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SO-04E Build/10.3.1.B.2.48) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.9
envolk[ITS]spider/1.6 (+http://www.envolk.com/envolkspider.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527934138-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528367905-0
googlebot2.1
Mozilla/5.0 (compatible; MJ12bot/v1.4.4; http://www.majestic12.co.uk/bot.php?+)
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SO-04E Build/10.1.1.D.2.31) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.2
+http://www.googlebot.com/bot.html
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/Nutch-1.14-SNAPSHOT
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SHL23 Build/S8201) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.22
boitho.com-dc/0.4 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (iPad; CPU iPhone OS 11_0_3 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/5.3.1
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) https://deepcrawl.com/bot
Mozilla/5.0 (compatible; Googlebot/2.5; +http://www.google.com/bot.html)
boitho.com-dc/0.79 ( http://www.boitho.com/dcbot.html )
Googlebot/2.1 (+http://www.googlebot.com/bot.html) <iframe src="http://www.whatismyip.com/"></iframe>
boitho.com-dc/0.81 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528568825-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528750495-0
boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (Macintosh; Intel Mac OS X 1082; facebookexternalhit)AppleWebKit/537.17 (KHTML like Gecko) Chrome/24.0.1309.0Safari/537.17
Mozilla/5.0 (Linux; Android 7.1.2; Robin Build/NJH47B; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.22
boitho.com-dc/0.83 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Wget/1.10.2 (via Wayback Save Page)
Googlebot/2.1 (+http:\\www.googlebot.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/5.6.1
Mozilla/5.0 (compatible; Charlotte/1.0b; charlotte@beta.spider.com)
Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Safari/8536.25 Googlebot/2.1 scarlet/1.0 (development)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528491471-0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPad; CPU OS 9_2_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13D11 Googlebot/2.1 scarlet/1.0 (development)
Googlebot/2.1 (+https://www.googlebot.com/bot.html)
Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.11) Gecko/20080109 (Charlotte/0.9t; http://www.searchme.com/support/) (Charlotte/0.9t; http://www.searchme.com/support/)
Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25 Googlebot/2.1 scarlet/1.0 (development)
Mozilla/5.0 (Linux; Android 5.0.2; HTC Butterfly s Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.89 Mobile Safari/537.36
AOL/9.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16B92 YJApp-IOS jp.co.yahoo.ipn.appli/4.13.2
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 (via Wayback Save Page)
Mozilla/5.0 (Windows NT 6.3; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0 (via Wayback Save Page)
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/40.0.0.24.199;]
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SOL21 Build/9.1.D.0.401) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.11
Mozilla/5.0 (Linux; Android 4.4.2; SOL23 Build/14.3.C.0.239) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.12
Mozilla/5.0 (Linux; Android 5.1; Cosmos V8-2 Build/LMY47D; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/56.0.2924.87 Mobile Safari/537.36
Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.1.0.0 Safari/537.36 Googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528823198-0
Mozilla/5.0 (Linux; Android 4.4.4; vivo Y27 Build/KTU84P) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 SogouSearch Android1.0 version3.0
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.1.0.0 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0; 501SO Build/32.1.D.0.386; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.98 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.news/2.5.1
BotThief (compatible; Googlebot/2.1; Location US)
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SO-04E Build/10.1.1.D.2.31) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.11.0
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.136 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.125 Mobile Safari/537.36
ConveraCrawler/0.9 (+http://www.authoritativeweb.com/crawl)
Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; HUAWEI G750-T00 Build/HuaweiG750-T00) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 SogouMSE,SogouMobileBrowser/3.8.2
Mozilla/5.0 (compatible; Yahoo Ad monitoring; https://help.yahoo.com/kb/yahoo-ad-monitoring-SLN24857.html) yahoo.adquality.lwd.desktop/1534713918-0
Mozilla/5.0 (X11; U; Linux armv7l; en-GB; rv:25.0; Presto/2.12.388) Gecko/20111216 Firefox/27.0 Googlebot/2.1 Maemo Browser 1.7.4.8 RX-51 N900
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527943484-0
ConveraCrawler/0.9c (+http://www.authoritativeweb.com/crawl)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Evolute5.9; JK5)
ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl)
Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2.3pre; Presto/2.12.388; Macintosh; Intel Mac OS X 10.6; rv:25.0) Gecko/20111216 Firefox/13.0.1 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Maemo Browser 1.7.4.8 RX-51 N900
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; N-04E Build/A1002601) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.4.1
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528951593-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1534714788-0
CovarioIDS/1.1 (http://www.covario.com/ids; support at covario dot com)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528825209-0
NokiaX2-01/5.0 (08.63) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; YandexBot/3.0; http://yandex.com/bots) UNTRUSTED/1.0
Mozilla/5.0 (iPad; CPU OS 9_0 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13A340 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.6
Mozilla/5.0 (Linux; U; Android 5.0; de-de; E2303 Build/26.1.A.3.111) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 SogouMSE,SogouMobileBrowser/4.2.15
Mozilla/5.0 (Linux; U; Android 6.0; Cosmos_V5 Build/MRA58K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/58.0.3029.83 Mobile Safari/537.36 OPR/18.0.2254.106542
B-l-i-t-z-B-O-T/16.8 (Ark Linux 4.2; da_DK;)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528825616-0
Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A5339e YahooMobileSearch/1.0 (com.yahoo.search2/7.0.3; iOS/12.0;; iPhone8,1; Apple;;; 1334x750;)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM303SH Build/S0032) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.7
DataparkSearch/4.35 ( http://www.dataparksearch.org/)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1; 360Spider
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528826529-0
Mozilla/7.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 4.1.1; HTC Butterfly Build/JRO03C) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19
Mozilla/5.0 (Linux; Android 4.4.2; KYY24 Build/108.1.2110) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.news/1.17.1
Mozilla/9.0 (compatible; Googlebot/3.7; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528308591-0
Mozilla/5.0 (Windows NT 5.1; U; Win64; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)
Mozilla/5.0 (compatible; archive.org_bot/3.3.0 +http://pandora.nla.gov.au/crawl.html)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM303SH Build/S0032) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.73
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SO-02E Build/10.3.1.B.0.256) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.4.1
Googlebot/2.1 ([www.googlebot.com])
Mozilla/5.0 (compatible; BeslistBot; nl; BeslistBot 1.0; http://www.beslist.nl/
Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0; BingPreview/1.0b) like Gecko
archive.org_bot
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529256786-0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SH-08E Build/S9021) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.9.1
Mozilla/5.0 (Linux; U; Android 4.3; zh-tw; HTC_Butterfly_s Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Googlebot (iPhone; X11; Linux x86_64; compatible; WOW64; Gecko/20130331; MSIE 10.0; Android 2.2; Windows NT 10.0) Firefox/49.0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Linux; Android 4.4.4; SH-02G Build/S4010) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.26
Mediapartners-Google Yandex.Translate
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; WOW64; Trident/5.0; BingPreview/1.0b)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528234604-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528235090-0
SECSSOBrowser (compatible; Googlebot/2.1; +https://sts.secsso.net)
EARTHCOM.info/1.x [www.earthcom.info]
Microsoft Live Labs Zoom.it <http://zoom.it/> (via Wayback Save Page)
Mozilla/5.0 (Linux; Android 7.0; VIE-AL10; Build/HUAWEIVIE-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.92 Mobile Safari/537.36 SogouMSE,SogouMobileBrowser/5.8.7
Googlebot (Google's spider)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528236733-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1529744473-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1526173298-0
Googlebot/2.1+
Mozilla/5.0 (Linux; Android 7.1.1; Nexus 5X Build/N4F26I; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/55.0.2883.91 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.25
Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; Butterfly_M1 Build/KOT49H) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1 UCBrowser/10.4.2.659 Mobile
Mozilla/5.0 (Linux; Android 5.0.2; 401SO Build/23.1.F.0.464; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/55.0.2883.91 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.18
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; F-02F Build/V21R66B) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.0.26
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528767282-0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; F-02F Build/V21R66B) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.35
envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528586441-0
googlebot/2.1 (+http //www.google.com/bot.html)
Opera/9.80 (J2ME/MIDP; Opera Mini/2.1 (compatible; MSIE 4.01; Windows CE; Smartphone; 320x240;) MSNBOT-MOBILE/22.414; U; en) Presto/2.5.25 Version/10.54
Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible ; Googlebot/2.1 ; +http://www.google.com/bot.html)
Mozilla/5.0 (Windows NT 9000; rv:9000.0) Gecko Firefox/9000.0 (via ggpht.com googleimageproxy)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1532512728-0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; 202F Build/V23R53C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.6
msnbot/2.0b v1367549217.4911
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528158161-0
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SO-04E Build/10.1.1.D.2.31) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.6
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527955985-0
Mozila/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; N-03E Build/A7202901) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.10
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527790623-0
Hidden/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
facebookexternalhit/1.1 (+http(s)://www.facebook.com/externalhit_uatext.php)
"Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)
Mozilla/6.0 (compatible; Googlebot/3.9; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528062904-0
Mozilla/5.0 (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)
Mozella/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
NebulaMozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/)
NokiaC1-01/2.0 (04.40) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.tapanwap.com/index.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.15 (KHTML, like Gecko; +http://www.diffbot.com) Chrome/24.0.1295.0 Safari/537.15
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528064037-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528160639-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528593310-0
Mozilla/5.0 (Linux; Android 5.0.2; HTC Butterfly s) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.80 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SGP412 Build/14.1.B.3.320) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.35
Mozilla/5.0 (Linux; U; Android 4.1.1; ja-jp; SC-03E Build/JRO03C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.11.0
Y!J-ASR/1.0 crawler (http://www.yahoo-help.jp/app/answers/detail/p/595/a_id/42716/)
Mozilla/5.0 (Linux; Android 4.1.2; SCH-R530U; Build/JZO54K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/46.0.2490.92 SDK/1.1.3.423 Mobile Safari/537.36 SogouMSE,SogouMobileBrowser/5.2.15
Mozilla/5.0 (Windows; U; Windows NT 9.5; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/534.19(KHTML,like Gecko) Chrome/11.0.661.0 Safari/534.19 UNTRUSTED/1.0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; KYY21 Build/209.0.1700) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.16
Googlebot/2.2 (+http://www.googlebot.com/bot.html)
Googlebot-Image/1.0 (Prism; XKeyscore)
(compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.myteltech.blogspot.in/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527723422-0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)/1.8 (InfoSeek crawler; http://www.infoseek.com; crawler@infoseek.com)
Googlebot/2.1 (+http://www.google.com/bot.html; For abuse: kilobyte2007@gmail.com)
Mozilla/5.0 (compatible; Googlebot/
Mozilla/5.0 (compatible; Baiduspider/2.0; http://www.baidu.com/search/spider.html)
Mozilla/5.0 (compatible; Googlebot/7.0
MSNBOT/0.1x (http://search.msn.com/msnbot.htm)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528247981-0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SO-04E Build/10.3.1.B.2.13) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.20
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527794904-0
Hacker Anonnymous (+http://www.googlebot.com/bot.html
Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527726830-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527796048-0
Accoona-Biz-Agent/1.1.1 crawler@accoona.com
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528598259-0
Accoona-AI-Agent/1.1.2
=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1;
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.78 Mobile Safari/537.36
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1535566467-0
GoogleBot/15.18 (Ubuntu 5.3; ar_JO;)
Mozilla/5.0 (compatible; U; Googlebot/9.8.4.62; + http://www.google.com/bot; en) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Mozilla/5.0 (iPad; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/5.1.0
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; LaVieTab PC-TE508S1 Build/JDQ39) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.29
Mozilla/5.0 (compatible; Googlebot/20.0.1105.2; + http://www.google.com/bot.html) UNTRUSTED/1.0
ia_archiver-web.archive.org
Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27 YJApp-IOS jp.co.yahoo.yjtrend01/5.1.1
Covario IDS/4.16 (Sabayon Linux 1.4; da;)
Mozilla/5.0 (Linux; Android 2.3.2; Nexus One; Googlebot/20.0.1105.2; + http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (compatible;Baiduspider/2.0;+http://www.baidu.com/search/spider.html)
Mozilla/5.0 (Linux; Android 5.0.2; SO-03G Build/28.0.B.0.488; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/45.0.2454.95 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.27
Twitterbot
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SH-05F Build/S3140) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.5.1
Mozilla/5.0 (Linux; Android 4.4.2; HTC_Butterfly_s_901s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.141 Mobile Safari/537.36
Googlebot/4.0 (MSIE 4.01; ExtremeOS 10_3_1 PPC; rv:12.6) Crocodile/20040327 Gridspider/12.6
Mozilla/5.0 (Linux; Android 5.0.2; HTC_Butterfly_s_901s Build/LRX22G; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.109 Mobile Safari/537.36
Google Chrome/5.0 (Linux; Android 8.0.0 Pixel XL Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
mcreighbaum@yahoo.com
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36 Mediapartners-Google
Google Chrome/61 (Linux; Android 8.0.0 Pixel XL Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529537669-0
BlitzBOT@tricus.net (Mozilla compatible)
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/46.0 Cookie: cookie_check=yes; LANG=en_US%3BUS; tsrce=settingsnodeweb; ts=vr%3D93131de515e0a1e1aa72afbafffcfa01%26vreXpYrS%3D1600376834%26vteXpYrS%3D1505707857%26vt%3D93131e0815e0a1e1aa72afbafffcfa00; nsid=s%3ARW4NtXaqAl1ZvHO3GThta76SEhm-kA3N.C%2BLI2qgxz3xSE16I3soXGi%2FjSnxf5%2FZ7ptPnRWQpb7I; AKDC=phx-origin-www-1.paypal.com; akavpau_ppsd=1505706690~id=e1675f8e8f266e7a4f157877c7d97bb1; xppcts=ff50e6de21071ce9; x-pp-s=eyJ0IjoiMTUwNTcwNjA3MzY4MiIsIm0iOiIwIn0; login_email=Shuntayrbell%40yahoo.com; ui_experience=home%3D2%26login_type%3DEMAIL_PASSWORD; fn_dt=b99b8b5f23be427d9acf28251294ddec; HaC80bwXscjqZ7KM6VOxULOB534=cFylYJRmgbGlNUra8xF4A2cBOsCJbcZUwx4U9W41abi3iFuQhRryxj1KxEauh5SE42M6XiNeTI1DX3DtGVLmLOty8qCmsge2JsdwnndJyTx0UnZNd9_rKNE_Ft_MybFQ6gc9sG; X-PP-ADS=AToBUUC.WVwpsocPdB1riDLAoHoEKRM; navcmd=_complaint-view; KHcl0EuY7AKSMgfvHl7J5E7hPtK=GECMmlTEa1-31thGJR-NcGDCX7y_VVgmZvFLJT1jperovxI9WbVlWCv8sJo6U9r4aaXsYnFuwFKjjn5u; x-pp-p=4PdpkSOYGmxzAXw6ouE5W.6CJ8MFiXwo6J9g1z6J.Vp43pu6gQ0lZ4WTpnf5JwBPT.Jj4TcKjE4yfDtxbCC5.wLlOSR1CXUuzI-30ZPytnk8roEIR-S.ExMxZYZyf7XZ6mpEKVF8SazU1eXDv8fszzk93MkhsYfEgGHv6FD.0.aS.QiwMc6Qxg4w2Ow6HVp4izfXJn9wuvE7M.-EAbpbAle.gJC8q2uVLNuhWR8sdn5V3nQJvz.43ycXPskM.oX2; SEGM=kPdr9wAD9vnD2klO5j9sDt3jd9Dr8DDbshd72DDrt8ZQpln941oplFGH9isom0wDWnGlL83Ghjir932dfkd8FCou; cwrClyrK4LoCV1fydGbAxiNL6iG=NdAmkcZOS7wshP4kfhiiH1FWm2TpM6ukxUHRvaUnVaKLoK0Rxzg9gcDZUT034k5aa7HY_R5ddGdOPz6FxrZv-wPHLjr9_glPFf2GaH0aspgUHC2iIsMFfu6FdIv2NGLmmKg9ujEAyEI6w6d8c1OAHXIa2Wp7rC6_-u1M6P0WjiKHqLB70ONkbPhUz6gmc_zIUZnbaEoebqZv61D3tViz1quzVECsxH-ngurvjFShtxeX19qjVETmCYOlQr4
Mozilla/5.0 (Linux; Android 5.0.2; SO-02G Build/23.1.B.1.317; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/54.0.2840.85 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.15
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527880481-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529537745-0
Mozilla/5.0 (Linux; U; Android 5.0.2; zh-cn; X800+ Build/BEXCNFN5501102031S) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 SogouMSE,SogouMobileBrowser/4.2.10
BlitzBOT@tricus.com (Mozilla compatible)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM303SH Build/S0032) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.4.1
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1528256718-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529537959-0
Mozilla/5.0 (iPad; CPU OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60 YJApp-IOS jp.co.yahoo.ipn.appli/4.11.5
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) support@google.com
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529105493-0
Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0 3gpp-gba
CatchBot/2.0
Mozilla/5.0 (Linux; Android 5.0.2; SH-05G Build/S3280; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/51.0.2704.81 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.17
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529539958-0
Mozilla/5.0 (Linux; Android 6.0; SO-05G Build/32.1.E.1.54; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.31
Mozilla/5.0 (iPad; U; ipad/12; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527801039-0
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) FirePHP/0.5
msn/1.0 (+http://search.msn.com/msnbot.htm)
Mozilla/5.0 (compatible; Yahoo Ad monitoring; https://help.yahoo.com/kb/yahoo-ad-monitoring-SLN24857.html) yahoo.adquality.lwd.desktop/1540573980-0
66.249.64.14 - crawl-66-249-64-14.googlebot.com
Mozilla/5.0 (Windows; U; Windows NT 10.0; en-US) AppleWebKit/604.1.38 (KHTML, like Gecko) QtWebEngine/5.10.1 googlebot/604.1.38
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1539436045-0
Mozilla/5.0 (Linux; Android 5.0.2; SHV32 Build/S7060; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/45.0.2454.95 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.0.25
Mozilla/5.0 (Windows NT 6.2; rv:30.0) Gecko/20150101 Firefox/32.0 360Spider 360Spider
Nokia2700c-2/2.0 (09.98) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (iPhone; U; CPU iPhone 0S 3.0 like Mac 0S X; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16/UCWEB8.2.0.116/69/352 UNTRUSTED/1.0
Mozilla/4.0 (Googlebot/2.1; MSIE 6.0; http://www.googlebot.com/bot.html)
BingBot (Bing's spider)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528329729-0
Mozilla/4.0 (http://www.googlebot.com/bot.html; Googlebot/2.1; )
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; FJL22 Build/V12R53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.13.1
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527804299-0
Mozilla/5.0 (Linux; Android 5.1.1; C6833 Build/14.6.A.1.236; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.30
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) https://deepcrawl.com/bot
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529543867-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1534902842-0
Googlebot/2.1 (Windoze; N; Windows 95; ZUSE3; rv1.3.10) Trident/20100101 Icefox/11.0b4
Mozilla/5.0 (Linux; U; Linux; ja-jp; DTV; TSBNetTV/T4101D9.0203.9DD) AppleWebKit/536(KHTML, like Gecko) NX/3.0 (DTV; HTML; R1.0;) DTVNetBrowser/2.2 (000039;T4101D9;0203;9DD) InettvBrowser/2.2 (000039;T4101D9;0203;9DD) YahooDTV/2.0 (Video=0x03;Audio=0x01;Screen=0x01;Device=0x00;Remote=0x10)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527887380-0
Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html
Googlebot-Image 1.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527888266-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527974735-0
Mozilla/5.0 (Linux; Android 5.1.1; Nexus 7 Build/LMY47V; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/50.0.2661.86 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.12
NokiaC1-01/2.0 (06.15) Profile/MIDP-2.1 Configuration/CLDC-1.1 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1540259578-0
Mozilla/5.0 (Linux; Android 6.0.1; HTC Butterfly 2 Build/MMB29M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.81 Mobile Safari/537.36
Googlebot-Image 2.1
Sogou+web+spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) UNTRUSTED/1.0 3gpp-gba
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527976918-0
You!! Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 (via Wayback Save Page)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528091376-0
Mozilla/5.0 (Linux; Android 4.4.4; SHV31 Build/S1190) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.25
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SH-01FDQ Build/SC100) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.news/1.17.1
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528091377-0
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1525134178-0
Mozilla/5.0 (Linux; Android 6.0; Cosmos V20 Build/K19_HOTWAV_COSMOS_V20_220; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.109 Mobile Safari/537.36
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528091378-0
Mozilla/5.0 (Linux; Android 4.4.2; 402LG Build/KOT49I.402LG10d) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.20
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528091379-0
Nokia7210Supernova/2.0 (07.23) Profile/MIDP-2.1 Configuration/CLDC-1.1 Googlebot/2.1#(+http://www.google.com/ bot.html)/UCWEB7.5.0.66/69/352 UNTRUSTED/1.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528523564-0
Mozilla/5.0 (iPhone; CPU OS 8_0 like Mac OS X) AppleWebKit/537.51.3 (KHTML, like Gecko) Version/8.0 Mobile/11A4132 Safari/9537.145 evaliant
Googlebot/2.0
Mozilla/5.0 (compatiable Googlebot/2.1 + http//www.Google.com/not html)
Googlebot/2.X (http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528092758-0
Mozilla/4.0_(compatible;_MSIE_5.0;_Windows_95)_VoilaBot/1.6 libwww/5.3.2
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15G77 webview/388515511fd67e09fa9f9ae29177028f YJApp-IOS jp.co.yahoo.apppkgcal/2.4.7
Mozilla/5.0 (Linux; Android 5.0.2; 402SO Build/28.0.C.3.33; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/46.0.2490.76 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.32
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM302SH Build/S0018) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.9
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528791579-0
MSNbot/1.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1523745114-0
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.15 Safari/537.36, : Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
msnbot/2.0b v1374263781.0484
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1524679868-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528335931-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528395176-0
Mozilla/5.0 (Linux; Android 4.4.2; F-02F Build/V11R22C) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.21
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528395177-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528464875-0
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529204208-0
Mozilla/5.0%20%28iPhone%3B%20CPU%20iPhone%20OS%206_0%20like%20Mac%20OS%20X%3B%20en-us%29%20AppleWebKit/536.26%20%28KHTML%2C%20like%20Gecko%29%20Version/6.0%20Mobile/10A5376e%20Safari/8536.25%20%28compatible%3B%20Googlebot-Mobile/2.1%3B%20%2Bhttp%3A//www.google.com/bot.html%29
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1537402177-0
Mozilla/5.0 (Linux; Android 4.4.2; LGL22 Build/KOT49I.LGL2220d) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.17
MJ12bot/vx.x.x (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php)
Mozilla/5.0 (Linux; Android 5.0.1; 503HW Build/HuaweiALE-L02; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/51.0.2704.81 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.0.8
Mozilla/5.0 (Linux; Android 6.0.1; MI MAX; Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.92 Mobile Safari/537.36 SogouMSE,SogouMobileBrowser/5.5.5
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528795673-0
Mozilla/5.0 (Windows; U; Windows NT 6.2; en-US; Valve Steam GameOverlay/1459463254; ) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 (via Wayback Save Page)
compatible;Baiduspider/2.0; +http://www.baidu.com/search/spider.html
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) .default/1538875462-0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) AlexaToolbar/alxf-2.18
Mozilla/5.0 (compatible;archive.org_bot/1.7.1; collectionId=316; Archive-It; +http://www.archive-it.org)
Mozilla/5.0 (Linux; Android 5.0.2; 403SH Build/S0022; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/42.0.2311.138 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.16
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM302SH Build/S0020) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.3
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528796845-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529422613-0
Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0) BingPreview/1.0b
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SH-04E Build/S5160) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.1.7
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) ASProxy/5.5.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36 Googlebot/2.1
FAST Enterprise Crawler 6 used by Schibsted (webcrawl@schibstedsok.no)
Mozilla/5.0 (Linux; U; Android 4.3; en-us; HTC_Butterfly Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528468212-0
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529208403-0
Googlebot/2.1 (+http://www.googlebot.com/bot.html (Windows NT 6.3; Win64; x64; Trident/7.0; rv:11.0) like Gecko
boitho.com-robot/1.x
Mozilla/5.0 (compatible; Googlebot[b]new[/b]/2.1; +http://www.google.com/bot.html)
mozilla/5.0 (compatible Googlebot/2.1 +http,//www.google.com/bot html)
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; F-01F Build/V19R43G) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.2
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528797944-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529209783-0
FAST EnterpriseCrawler 6
Googlebot 2.1(+http://www.googlebot.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528468651-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529210262-0
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1527750292-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528468831-0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529210263-0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Trident/5.0)
facebookexternalhit/1.1 (+http(s)://www.facebook.com/externalhit_uatext.php
Mozilla/5.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528101414-0
Mozilla/5.0 (compatible; Googlebot/3333; +http://www.google.com/bot.html)
Mozilla/4.0 (hhjhj@yahoo.com)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528274535-0
msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; ) Firefox/1.5.0.11; 360Spider
Mozilla/5.0 (Linux; Android 7.0; S2 Build/1.010VE.42.a; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/58.0.3029.83 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.14
FAST FreshCrawler 6
Mozilla/5.0 (NokiaC5-00; compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1537406187-0
Googlebot/2.1 (+http://www.googlebot.com/bot.html) AlexaToolbar/alxf-2.19
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529211103-0
Mozilla/5.0 (Linux; U; Android) Googlebot/2.1
Mozilla/5.0 (Linux; U; Android 4.2.2; zh-tw; HTC_Butterfly Build/JDQ39) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (Linux; Android 4.4.2; LGL22 Build/KOT49I.LGL2220d) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.21
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528799526-0
Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/) ASProxy/5.5b4
Mozilla/5.0 (Linux; Android 4.4.2; F-01F Build/V10R22A) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.9
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp )
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1540190131-0
Mozilla/5.0 (Linux; Android 4.4.2; LGL22 Build/KOT49I.LGL2220d) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.22
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529212003-0
Mozilla/5.0 (compatible; Googlebot<b>New</b>/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_5 like Mac OS X) AppleWebKit/604.5.3 (KHTML, like Gecko) Mobile/15D5049a Sogousearch/Ios/5.9.5
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; F-07E Build/V21R40B) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.3.21
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0; WOW64; Trident/6.0; BingPreview/1.0b)
Mozilla/5.0 (Linux; Android 8.1.0; S3-SH Build/S0110; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.27.0
Mozilla/5.0 (compatible; Bixo/Test; +http://wiki.github.com/bixo/bixo/bixocrawler; bixo-dev@yahoogroups.com)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 4.3; HTC Butterfly Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.114 Mobile Safari/537.36
Googlebot-Mobile/2.1
Mozilla/5.0 (Linux; U; Android 4.2.2; ja-jp; SBM303SH Build/S0032) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.5.9
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528276115-0
Mozilla/5.0 (Android; Android; en-us; compatible; Googlebot/2.1; http://www.google.com/bot.html; AppleWebKit/528.18(KHTML,like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 UNTRUSTED/1.0
Mozilla/5.0 (Linux; Android 4.4.2; SO-03F Build/17.1.1.B.2.73) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/2.4.13
Mozilla/5.0 (Linux; U; Android 4.1.2; ja-jp; SBM203SH Build/S0030) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.6
Mozilla/5.0 (Twitterbot/0.1)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529212605-0
Mozilla/5.0 (Android; U; Android; en-id; compatible; Googlebot 10.0) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529212609-0
Googlebot/Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0
Mozilla/5.0 (Linux; Android 6.0; Cosmos V20 Build/K19_HOTWAV_COSMOS_V20_168; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 5.0.2; JCI VA-10J Build/LRX22G-2035; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.109 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.17.0
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529212611-0
Mozilla/5.0 (Linux; Android 4.4.4; 402ZT Build/KTU84P) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/1.7.24
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1540192574-0
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36
Googlebot (Windows NT 6.1; Win64; x64)
Mozilla/5.0 (Android; U; Googlebot/2.1; en-id) AppleWebKit/534.1+ (KHTML, Like Gecko) Version/6.0.0.141 Mobile Safari/534.1+ UNTRUSTED/1.0
[Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) or (rarely used): Googlebot/2.1 (+http://www.google.com/bot.html)]
Mozilla/5.0 (Linux; U; Android 4.0.3; ja-jp; KFTT Build/IML74K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.29
Mozilla/5.0 (Linux; Android 5.0.2; SOT21 Build/23.1.G.2.151; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/54.0.2840.68 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.ybrowser/2.0.20
Mozilla/5.0 (iPhone; CPU iPhone OS 10_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176 YJApp-IOS jp.co.yahoo.ymail/3.12.4
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528110586-0
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12F70 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.15
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1536294810-0
Mozilla/5.0 (compatible; <B>Googlebot</B>/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.9oo91e.qjz9zk/bot.html)
Mozilla/5.0 (Linux; U; Android 4.4.2; zh-tw; HTC_Butterfly_s Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
admantx-testex/2.4 (+http://www.admantx.com/service-fetcher.html)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0 (via Wayback Save Page)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1538100594-0
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; Android 4.0.3; ja-jp; KFTT Build/IML74K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.1.26
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 (via Wayback Save Page)
Mozilla/5.0 (compatible; bingbot/2.0; http://www.bing.com/bingbot.htm)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) sieve-gq1/1538513527-0
Mozilla/5.0 (compatible; Googlebot/2.1; +http://google.com/)
Mozilla/5.0 (compatible; Googlebot/2.0; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1528278088-0
Mozilla/5.0 (compatible; Googlebot/2.2; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.3; tr-tr; HTC_Butterfly Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (compatible; Googlebot/2.1; ̣+http://www.google.com/bot.html)
Mozilla/5.0 (Linux; U; Android 4.4.2; ja-jp; HTC_Butterfly Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (iPad; CPU OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36 YJApp-IOS jp.co.yahoo.ipn.appli/4.6.9
Mozilla/5.0 (compatible; Yahoo! DE Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/4.0 (compatible; Googlebot-Mobile 2.1; S60; SymbOS; Opera Mini/8.0.35626; U; en) Chrome/8.0
Mozilla/5.0 (Linux; Android 5.0.2; HTC_Butterfly_s Build/LRX22G) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/38.0.0.0 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 4.4.2; HTC Butterfly s Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.114 Mobile Safari/537.36
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) kafe/1529371453-2
Mozilla/5.0 (compatible; Yahoo Ad monitoring; https://help.yahoo.com/kb/yahoo-ad-monitoring-SLN24857.html) yahoo.adquality.lwd.desktop/1534935443-0
Mozilla/5.0 (Linux; Android 4.4.2; F-01F Build/V10R22A) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/3.2.15
Mozilla/5.0 (compatible; Googlebot/2.1; +http//www.google.com/bot.html) (compatible; MSIE 6.0; Windows CE; IEMobile 7.6)
Mozilla/5.0 (Linux; U; Android 4.2.2; ru-ru; HTC_Butterfly_s Build/JDQ39) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (Linux; Android 4.4.4; d-01G Build/HuaweiMediaPad) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.1750.136 Safari/537.36 YJApp-ANDROID jp.co.yahoo.android.yjtop/10.0.34