How does Yahoo/Google find unknown domains?

Michelle Konzack linux4michelle at tamay-dogan.net
Wed Nov 10 00:47:44 UTC 2010


Hello Robert Spangler,

Am 2010-11-09 10:34:52, hacktest Du folgendes herunter:
> If these domains are for internal use only, why did you list the DNS servers 
> for them?  You are aware that you can register a domain without listing a DNS 
> Server?

Because my own customers (exclusively) must access it.  They are my VOIP
and IPTV servers and there is no public HTTP content  but  I  am  bombed
with PHP/CGI requests and I do not know, where Google and Co have gotten
those links.

Some of the searchbots are hiting my servers  3  times  in  series  from
three different IPs and in summary, I have more then  10.000  searchbot-
entries per day in my Logs.

My server always return an Error-Page from the VServer that there is  no
configured HTTP host on the machine but it is ignored.  One  VHost  must
be configured for the web administration and it is hit too and too much!

Even my simple squirrelmal login page from <webmail.tamay-dogan.net>  is
spidered daily with more then 800 hits and I have already  counted  more
then 80 different searchbots.

How braindamaged are Searchbot-Programmers?

All of my webservers together have arround 86 TByte of content including
a VERY huge debian archive (all releases and versions from 0.96 to  now)
and my traffic per month is arround 27 TByte.

The searchbots  are  creating 17,3 TByte  traffic  per  month  which  my
customers have to pay to!

Maybe I call my lawer to write letters to the serachbot owners  to  stop
spidering my 36 domains.

Oh, at Level3 in Frankfurt I pay 12 Euro/Mbit traffic  per  month  which
mean 12 Euro per 320 GByte traffic. Not counting the price for the 700km
FiberOptic line which is another provider (0,40 euro/m/year).

I had a 1 GE line from Frankfurt but du to the excessiv serchbot traffic
it broke several times per day.  Now I have in total twelf 1GE  (Level3,
Verizon, DTag and  Orange).  Maximal  I  can  have  64 x 10 GE  with  my
Transmode TS System but then I can install my own BPOP.

Thanks, Greetings and nice Day/Evening
    Michelle Konzack

-- 
##################### Debian GNU/Linux Consultant ######################
   Development of Intranet and Embedded Systems with Debian GNU/Linux

itsystems at tdnet France EURL       itsystems at tdnet UG (limited liability)
Owner Michelle Konzack            Owner Michelle Konzack

Apt. 917 (homeoffice)
50, rue de Soultz                 Kinzigstraße 17
67100 Strasbourg/France           77694 Kehl/Germany
Tel: +33-6-61925193 mobil         Tel: +49-177-9351947 mobil
Tel: +33-9-52705884 fix

<http://www.itsystems.tamay-dogan.net/>  <http://www.flexray4linux.org/>
<http://www.debian.tamay-dogan.net/>         <http://www.can4linux.org/>

Jabber linux4michelle at jabber.ccc.de
ICQ    #328449886

Linux-User #280138 with the Linux Counter, http://counter.li.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.pgp
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20101110/9150f110/attachment.bin>


More information about the bind-users mailing list