You can upload your custom robot.txt via metadata API. Here are the steps:

1. Install the force.com ide

2. Create a Visualforce Page that represents the robots.txt you would like

<apex:page contentType="text/plain" showHeader="false">
User-agent: msnbot
Disallow: /
</apex:page>

3. Add the following to your package.html

<types>
<members>*</members>
<name>CustomSite</name>
</types>

4. "Refresh from Server" before you can see the sites.

5. Open up your site under the "sites" folder and add in a "robotsTxtPage" entry and save.

<?xml version="1.0" encoding="UTF-8"?>
<CustomSite xmlns="http://soap.sforce.com/2006/04/metadata">
<active>true</active>
<authorizationRequiredPage>Unauthorized</authorizationRequiredPage>
<bandwidthExceededPage>BandwidthExceeded</bandwidthExceededPage>
<changePasswordPage>ChangePassword</changePasswordPage>
<fileNotFoundPage>FileNotFound</fileNotFoundPage>
<inMaintenancePage>InMaintenance</inMaintenancePage>
<indexPage>IdeasHome</indexPage>
<masterLabel>First</masterLabel>
<portal>Customer Portal</portal>
<requireInsecurePortalAccess>false</requireInsecurePortalAccess>
<robotsTxtPage>RobotsTxt</robotsTxtPage>
<siteAdmin>mylogin@myco.com</siteAdmin>
<subdomain>mydomain</subdomain>
</CustomSite>

6. Now http://mydomain.force.com/robots.txt will contains the description of your page


You can also use your own custom favorite icon. Here are the steps:

1. create 16x16 icon
2. upload as a static resource and make sure cache control attribute is set as public
3. Disable the standard header in your visualforce page and reference the favorite icon static resource in your page as following:

<apex:page showHeader="false" >
...
...
<link REL="SHORTCUT ICON" HREF="{!URLFOR($Resource.favicon)}">
.....
</apex:page>