How does Yahoo/Google find unknown domains?

[Michelle Konzack]

Hello Sten Carlsen,

Am 2010-11-08 02:32:14, hacktest Du folgendes herunter:
> Did you consider robots.txt? Well behaved spiders should respect that,
> although it does not prevent anything.

It is a VHost without own doc_root, which mean, a robots.txt would block
anything on the Server

> I would expect google and yahoo to respect it, I have no clue about he.net.

I know, but it is not possibel without instaliing a separated server.

Thanks, Greetings and nice Day/Evening
    Michelle Konzack

-- 
##################### Debian GNU/Linux Consultant ######################
   Development of Intranet and Embedded Systems with Debian GNU/Linux

itsystems at tdnet France EURL       itsystems at tdnet UG (limited liability)
Owner Michelle Konzack            Owner Michelle Konzack

Apt. 917 (homeoffice)
50, rue de Soultz                 Kinzigstraße 17
67100 Strasbourg/France           77694 Kehl/Germany
Tel: +33-6-61925193 mobil         Tel: +49-177-9351947 mobil
Tel: +33-9-52705884 fix

<http://www.itsystems.tamay-dogan.net/>  <http://www.flexray4linux.org/>
<http://www.debian.tamay-dogan.net/>         <http://www.can4linux.org/>

Jabber linux4michelle at jabber.ccc.de
ICQ    #328449886

Linux-User #280138 with the Linux Counter, http://counter.li.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.pgp
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20101108/cccf9c8e/attachment.bin>