Question about visibility

Barry Margolin barmar at alum.mit.edu
Thu Oct 11 19:39:55 UTC 2018


In article <mailman.671.1539286015.803.bind-users at lists.isc.org>,
 Dennis Clarke <dclarke at blastwave.org> wrote:

> On 10/11/2018 03:21 PM, Leonardo Rodrigues wrote:
> > Em 11/10/18 16:13, Barry Margolin escreveu:
> >>
> >> If you accidentally, or someone else intentionally, create a link to the
> >> site that uses the IP and put it on a web page that Google can get to,
> >> it will probably find the page.
> >>
> >>
> > 
> >      robots.txt, on your website root, is your friend. Simply deny web 
> > crawling on it, and you're (probably) done.
> > 
> 
> If you believe robots.txt means anything at all.

Google is known to obey it, and the question was about avoiding getting 
your site indexed by Google.

Of course, that doesn't mean someone won't find the site on their own. 
If the link to it is on some other page that isn't blocked by 
robots.txt, someone might stuble across that page and then click on the 
link.

But if you're mainly worried about someone googling the words that are 
on your website and Google sending them to the development version 
instead of the production version, you're pretty safe.

Actually, DNS has very little impact on this at all. AFAIK, Google 
doesn't crawl DNS, it just crawls web pages and follows links. My 
company's development server is in DNS, and it's not firewalled (we all 
work from our homes, there's no company network to restrict access 
with), but I've never heard of anyone accidentally being directed there 
by Google, because we don't publish links to this server.

-- 
Barry Margolin
Arlington, MA


More information about the bind-users mailing list