Question about visibility

[N6ghost]

On Thu, 11 Oct 2018 15:39:55 -0400
Barry Margolin <barmar at alum.mit.edu> wrote:

> In article <mailman.671.1539286015.803.bind-users at lists.isc.org>,
>  Dennis Clarke <dclarke at blastwave.org> wrote:
> 
> > On 10/11/2018 03:21 PM, Leonardo Rodrigues wrote:  
> > > Em 11/10/18 16:13, Barry Margolin escreveu:  
> > >>
> > >> If you accidentally, or someone else intentionally, create a
> > >> link to the site that uses the IP and put it on a web page that
> > >> Google can get to, it will probably find the page.
> > >>
> > >>  
> > > 
> > >      robots.txt, on your website root, is your friend. Simply
> > > deny web crawling on it, and you're (probably) done.
> > >   
> > 
> > If you believe robots.txt means anything at all.  
> 
> Google is known to obey it, and the question was about avoiding
> getting your site indexed by Google.
> 
> Of course, that doesn't mean someone won't find the site on their
> own. If the link to it is on some other page that isn't blocked by 
> robots.txt, someone might stuble across that page and then click on
> the link.
> 
> But if you're mainly worried about someone googling the words that
> are on your website and Google sending them to the development
> version instead of the production version, you're pretty safe.
> 
> Actually, DNS has very little impact on this at all. AFAIK, Google 
> doesn't crawl DNS, it just crawls web pages and follows links. My 
> company's development server is in DNS, and it's not firewalled (we
> all work from our homes, there's no company network to restrict
> access with), but I've never heard of anyone accidentally being
> directed there by Google, because we don't publish links to this
> server.
> 

robot.txt is suppose to govern whats indexed... not sure how well its
followed nowadays but thats the process for it.