As described here: https://releasenotes.docs.salesforce.com/en-us/summer15/release-notes/rn_service_emails_case_feed.htm ,
it should be possible to control the value of the "IsExternallyVisible" flag through a trigger.

We have our own logic which works well and were executing this code in the "before insert" of an EmailMessage record.
We recently noticed that even while our code sets the value in the before insert, Salesforce overrides this value later. (between the before insert and after insert)...

To fix this, I moved the code to update this field to the "after insert" logic.
But doing so, we now get an error when trying to update the EmailMessage records "Invalid operation".
I don't re-use the records comming from the Trigger.isNew cause that would cause "read only" exceptions.
What I do is like this:

EmailMessage emailToUpdate = new EmailMessage();
emailToUpdate.Id = mId;
emailToUpdate.IsExternallyVisible = value;
emailsToUpdate.add(emailToUpdate);

So I'm sure I'm only updating the IsExternallyVisible flag and not any other field that I wouldn't be allowed to...
I used this code already in execute anonymous mode to fix some emails incorrectly flagged as externally visible and this works from there.

What am I missing? Why doesn't there seem to be a proper way to control the IsExternallyVisible flag through code / trigger?...