How does Yahoo/Google find unknown domains?
linux4michelle at tamay-dogan.net
Wed Nov 10 00:47:44 UTC 2010
Hello Robert Spangler,
Am 2010-11-09 10:34:52, hacktest Du folgendes herunter:
> If these domains are for internal use only, why did you list the DNS servers
> for them? You are aware that you can register a domain without listing a DNS
Because my own customers (exclusively) must access it. They are my VOIP
and IPTV servers and there is no public HTTP content but I am bombed
with PHP/CGI requests and I do not know, where Google and Co have gotten
Some of the searchbots are hiting my servers 3 times in series from
three different IPs and in summary, I have more then 10.000 searchbot-
entries per day in my Logs.
My server always return an Error-Page from the VServer that there is no
configured HTTP host on the machine but it is ignored. One VHost must
be configured for the web administration and it is hit too and too much!
Even my simple squirrelmal login page from <webmail.tamay-dogan.net> is
spidered daily with more then 800 hits and I have already counted more
then 80 different searchbots.
How braindamaged are Searchbot-Programmers?
All of my webservers together have arround 86 TByte of content including
a VERY huge debian archive (all releases and versions from 0.96 to now)
and my traffic per month is arround 27 TByte.
The searchbots are creating 17,3 TByte traffic per month which my
customers have to pay to!
Maybe I call my lawer to write letters to the serachbot owners to stop
spidering my 36 domains.
Oh, at Level3 in Frankfurt I pay 12 Euro/Mbit traffic per month which
mean 12 Euro per 320 GByte traffic. Not counting the price for the 700km
FiberOptic line which is another provider (0,40 euro/m/year).
I had a 1 GE line from Frankfurt but du to the excessiv serchbot traffic
it broke several times per day. Now I have in total twelf 1GE (Level3,
Verizon, DTag and Orange). Maximal I can have 64 x 10 GE with my
Transmode TS System but then I can install my own BPOP.
Thanks, Greetings and nice Day/Evening
##################### Debian GNU/Linux Consultant ######################
Development of Intranet and Embedded Systems with Debian GNU/Linux
itsystems at tdnet France EURL itsystems at tdnet UG (limited liability)
Owner Michelle Konzack Owner Michelle Konzack
Apt. 917 (homeoffice)
50, rue de Soultz Kinzigstraße 17
67100 Strasbourg/France 77694 Kehl/Germany
Tel: +33-6-61925193 mobil Tel: +49-177-9351947 mobil
Tel: +33-9-52705884 fix
Jabber linux4michelle at jabber.ccc.de
Linux-User #280138 with the Linux Counter, http://counter.li.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 189 bytes
Desc: Digital signature
More information about the bind-users