+ Start a Discussion
charlie_vcharlie_v 

Robots.txt file won't refresh

Hi there. 

 

This morning I finally got around to attaching the Site Robots.txt file to my site.  A quick check of Google Webmaster Tools verified it was there, and they were seeing it.

 

Unfortunately, I inadvertently set all user agents to Disallow instead of Allow.  So I made the correction, but now that change doesn't seem to be propagating to the web.  I've waited 45 minutes, and it's still not out on the web.  I've verified the change is correct using the "preview" button in my Sites configuration.

 

How long before the updated robots.txt file gets out on the web?

Best Answer chosen by Admin (Salesforce Developers) 
BulentBulent

updates on robot.txt might take up to a day since the txt file would be cached for a day like static resources (the difference is that this file can't have a time stamp so your updates can't be reflected immediately)