I have logged a case but wanted to see what the community could figure out as well.

I have a set of business hours defined with no Sunday and no Saturday hours. The rest of the days have a 24 hour schedule. I am testing the code below in the developer console to make sure it works correctly. I am getting inconsistent results. I am using a start date of 3/27/2013 and trying to add 3 business days in one case and subtract 3 business days in another. This should give me a result of 4/1/2013 in the addition case and 3/22/2013 in the subtraction case. This is using the assumption that there are no business hours on Saturday and Sunday. What is actually happening is in the addition case as long as I use a starting time of 1 minute after midnight it works fine. If I set to anything less than that like midnight exactly it gives me an incorrect result. In the case of subtraction is is actually giving me a date of 3/23/2013 which is a Saturday which is not correct since there are no business hours on Saturday. Not sure what is going on here but this is not coming up with the correct date in both cases.

Run this in the developer console:
BusinessHours stdBusinessHours = [select id from businesshours where name = 'Name of Your Business Hours record with no hours on Saturday and Sunday and 24 hour days the rest of the week'];

if ((stdBusinessHours != NULL)) {
DateTime dt = Datetime.newInstance(2013, 3, 27, 0 , 1, 0);
System.debug(dt);
System.debug(BusinessHours.addGMT (stdBusinessHours.id, dt, 3 * 24 * 60 * 60 * 1000L).format());
System.debug(BusinessHours.addGMT (stdBusinessHours.id, dt, -3 * 24 * 60 * 60 * 1000L).format());
}

Results:

27.0 APEX_CODE,FINEST;APEX_PROFILING,INFO;CALLOUT,INFO;DB,INFO;SYSTEM,DEBUG;VALIDATION,INFO;VISUALFORCE,INFO;WORKFLOW,INFO
Execute Anonymous: BusinessHours stdBusinessHours = [select id from businesshours where name = 'ProcessComposer'];
Execute Anonymous:
Execute Anonymous: if ((stdBusinessHours != NULL)) {
Execute Anonymous: DateTime dt = Datetime.newInstance(2013, 3, 27, 0 , 1, 0);
Execute Anonymous: System.debug(dt);
Execute Anonymous: System.debug(BusinessHours.addGMT (stdBusinessHours.id, dt, 3 * 24 * 60 * 60 * 1000L).format());
Execute Anonymous: System.debug(BusinessHours.addGMT (stdBusinessHours.id, dt, -3 * 24 * 60 * 60 * 1000L).format());
Execute Anonymous: }
17:35:16.034 (34364000)|EXECUTION_STARTED
17:35:16.034 (34377000)|CODE_UNIT_STARTED|[EXTERNAL]|execute_anonymous_apex
17:35:16.034 (34494000)|VARIABLE_SCOPE_BEGIN|[1]|stdBusinessHours|BusinessHours|true|false
17:35:16.034 (34998000)|HEAP_ALLOCATE|[EXTERNAL]|Bytes:11
17:35:16.035 (35207000)|STATEMENT_EXECUTE|[1]
17:35:16.035 (35214000)|STATEMENT_EXECUTE|[1]
17:35:16.035 (35227000)|HEAP_ALLOCATE|[1]|Bytes:59
17:35:16.035 (35242000)|HEAP_ALLOCATE|[1]|Bytes:4
17:35:16.035 (35581000)|SOQL_EXECUTE_BEGIN|[1]|Aggregations:0|select id from businesshours where name = 'ProcessComposer'
17:35:16.040 (40857000)|SOQL_EXECUTE_END|[1]|Rows:1
17:35:16.040 (40880000)|HEAP_ALLOCATE|[1]|Bytes:8
17:35:16.040 (40896000)|HEAP_ALLOCATE|[1]|Bytes:29
17:35:16.041 (41010000)|HEAP_ALLOCATE|[1]|Bytes:8
17:35:16.041 (41027000)|HEAP_ALLOCATE|[1]|Bytes:39
17:35:16.041 (41117000)|HEAP_ALLOCATE|[1]|Bytes:8
17:35:16.041 (41171000)|VARIABLE_ASSIGNMENT|[1]|stdBusinessHours|{"serId":1,"value":{"Id":"01m70000000Tw2DAAS"}}|0x3c719eea
17:35:16.041 (41183000)|STATEMENT_EXECUTE|[3]
17:35:16.041 (41187000)|STATEMENT_EXECUTE|[4]
17:35:16.041 (41220000)|SYSTEM_METHOD_ENTRY|[4]|Datetime.newInstance(Integer, Integer, Integer, Integer, Integer, Integer)
17:35:16.041 (41255000)|SYSTEM_METHOD_EXIT|[4]|Datetime.newInstance(Integer, Integer, Integer, Integer, Integer, Integer)
17:35:16.041 (41274000)|VARIABLE_SCOPE_BEGIN|[4]|dt|Datetime|false|false
17:35:16.041 (41351000)|VARIABLE_ASSIGNMENT|[4]|dt|"2013-03-27T05:01:00.000Z"
17:35:16.041 (41360000)|STATEMENT_EXECUTE|[5]
17:35:16.041 (41385000)|SYSTEM_METHOD_ENTRY|[5]|System.debug(ANY)
17:35:16.041 (41410000)|USER_DEBUG|[5]|DEBUG|2013-03-27 05:01:00
17:35:16.041 (41420000)|SYSTEM_METHOD_EXIT|[5]|System.debug(ANY)
17:35:16.041 (41424000)|STATEMENT_EXECUTE|[6]
17:35:16.041 (41569000)|HEAP_ALLOCATE|[6]|Bytes:12
17:35:16.041 (41587000)|HEAP_ALLOCATE|[6]|Bytes:12
17:35:16.041 (41652000)|SYSTEM_MODE_ENTER|false
17:35:16.041 (41670000)|HEAP_ALLOCATE|[6]|Bytes:5
17:35:16.041 (41762000)|SYSTEM_METHOD_ENTRY|[6]|system.BusinessHours.addGmt(Id, Datetime, Long)
17:35:16.042 (42772000)|SYSTEM_METHOD_EXIT|[6]|system.BusinessHours.addGmt(Id, Datetime, Long)
17:35:16.042 (42785000)|SYSTEM_MODE_EXIT|false
17:35:16.042 (42817000)|SYSTEM_METHOD_ENTRY|[6]|Datetime.format()
17:35:16.042 (42882000)|HEAP_ALLOCATE|[6]|Bytes:17
17:35:16.042 (42892000)|SYSTEM_METHOD_EXIT|[6]|Datetime.format()
17:35:16.042 (42905000)|SYSTEM_METHOD_ENTRY|[6]|System.debug(ANY)
17:35:16.042 (42911000)|USER_DEBUG|[6]|DEBUG|4/1/2013 12:01 AM
17:35:16.042 (42919000)|SYSTEM_METHOD_EXIT|[6]|System.debug(ANY)
17:35:16.042 (42924000)|STATEMENT_EXECUTE|[7]
17:35:16.042 (42943000)|HEAP_ALLOCATE|[7]|Bytes:12
17:35:16.042 (42952000)|HEAP_ALLOCATE|[7]|Bytes:12
17:35:16.042 (42998000)|SYSTEM_MODE_ENTER|false
17:35:16.043 (43007000)|HEAP_ALLOCATE|[7]|Bytes:5
17:35:16.043 (43020000)|SYSTEM_METHOD_ENTRY|[7]|system.BusinessHours.addGmt(Id, Datetime, Long)
17:35:16.043 (43151000)|SYSTEM_METHOD_EXIT|[7]|system.BusinessHours.addGmt(Id, Datetime, Long)
17:35:16.043 (43161000)|SYSTEM_MODE_EXIT|false
17:35:16.043 (43180000)|SYSTEM_METHOD_ENTRY|[7]|Datetime.format()
17:35:16.043 (43197000)|HEAP_ALLOCATE|[7]|Bytes:18
17:35:16.043 (43204000)|SYSTEM_METHOD_EXIT|[7]|Datetime.format()
17:35:16.043 (43217000)|SYSTEM_METHOD_ENTRY|[7]|System.debug(ANY)
17:35:16.043 (43222000)|USER_DEBUG|[7]|DEBUG|3/23/2013 12:01 AM
17:35:16.043 (43229000)|SYSTEM_METHOD_EXIT|[7]|System.debug(ANY)
17:35:16.294 (43270000)|CUMULATIVE_LIMIT_USAGE
17:35:16.294|LIMIT_USAGE_FOR_NS|(default)|
Number of SOQL queries: 1 out of 100
Number of query rows: 1 out of 50000
Number of SOSL queries: 0 out of 20
Number of DML statements: 0 out of 150
Number of DML rows: 0 out of 10000
Number of code statements: 5 out of 200000
Maximum heap size: 0 out of 6000000
Number of callouts: 0 out of 10
Number of Email Invocations: 0 out of 10
Number of fields describes: 0 out of 100
Number of record type describes: 0 out of 100
Number of child relationships describes: 0 out of 100
Number of picklist describes: 0 out of 100
Number of future calls: 0 out of 10

17:35:16.294|CUMULATIVE_LIMIT_USAGE_END

17:35:16.043 (43304000)|CODE_UNIT_FINISHED|execute_anonymous_apex
17:35:16.043 (43311000)|EXECUTION_FINISHED