<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Michelle Konzack wrote:
<blockquote cite="mid:20101110004744.GM5441@michelle1" type="cite">
<pre wrap="">Hello Robert Spangler,
Am 2010-11-09 10:34:52, hacktest Du folgendes herunter:
</pre>
<blockquote type="cite">
<pre wrap="">If these domains are for internal use only, why did you list the DNS servers
for them? You are aware that you can register a domain without listing a DNS
Server?
</pre>
</blockquote>
<pre wrap=""><!---->
Because my own customers (exclusively) must access it. They are my VOIP
and IPTV servers and there is no public HTTP content but I am bombed
with PHP/CGI requests and I do not know, where Google and Co have gotten
those links.
</pre>
</blockquote>
Are you sure it's all search bots and not script kiddies playing?
Having a valid index.html in place that redirects to your business home
page may help. But the search bots don't normally search around for
php/cgi scripts. That however is typical of the script kiddies looking
for a server to hack.<br>
<blockquote cite="mid:20101110004744.GM5441@michelle1" type="cite">
<pre wrap="">
Some of the searchbots are hiting my servers 3 times in series from
three different IPs and in summary, I have more then 10.000 searchbot-
entries per day in my Logs.
My server always return an Error-Page from the VServer that there is no
configured HTTP host on the machine but it is ignored. One VHost must
be configured for the web administration and it is hit too and too much!
Even my simple squirrelmal login page from <webmail.tamay-dogan.net> is
spidered daily with more then 800 hits and I have already counted more
then 80 different searchbots.
How braindamaged are Searchbot-Programmers?
All of my webservers together have arround 86 TByte of content including
a VERY huge debian archive (all releases and versions from 0.96 to now)
and my traffic per month is arround 27 TByte.
The searchbots are creating 17,3 TByte traffic per month which my
customers have to pay to!
Maybe I call my lawer to write letters to the serachbot owners to stop
spidering my 36 domains.
Oh, at Level3 in Frankfurt I pay 12 Euro/Mbit traffic per month which
mean 12 Euro per 320 GByte traffic. Not counting the price for the 700km
FiberOptic line which is another provider (0,40 euro/m/year).
I had a 1 GE line from Frankfurt but du to the excessiv serchbot traffic
it broke several times per day. Now I have in total twelf 1GE (Level3,
Verizon, DTag and Orange). Maximal I can have 64 x 10 GE with my
Transmode TS System but then I can install my own BPOP.
Thanks, Greetings and nice Day/Evening
Michelle Konzack
</pre>
</blockquote>
Despite how I feel about Yahoo's SLURP engine, it still honors
robots.txt. Script kiddies don't.<br>
<br>
Lyle Giese<br>
LCR Computer Services, Inc.<br>
<br>
<br>
P.S. My last post on this. This is not DNS related.<br>
</body>
</html>