If you separate byte ports from character ports, and separate input
ports from output ports (at least at the API level), you get an easily
type-checked interface.  e.g.

open-input-file string [encoding keywords] -> input-character-port
read-char input-char-port -> character
open-input-file-raw string -> input-byte-port
read-byte input-byte-port -> integer

For your bidi ports, perhaps

open-input-output-file string [encoding keywords] -> input-char-port
output-char-port

with the two ports sharing common buffer structure in the implementation.

Often one needs to open a file or a structure initially as a byte port,
then decode subsequent sections of the sequence as characters of a
particular encoding.  For that, a procedure like

cook-input-encoding integer input-byte-port [encoding keywords] ->
input-char-port

can return a port that promises to decode a certain number of octets
from the backing byte port with your encoding.  It does't handle
variable-length structures well, though.

I like your read-substring and write-substring.