It is part of the spec, as noted in two places, in the description of as-unicode:

  Padding, trimming and tabbing, etc. will generally not

  do the right thing in the presence of zero-width and

  double-width Unicode characters, or ANSI escapes

and in item 6 of the unicode-terminal-width spec:

  ANSI terminal escapes, as output by the color formatters above, count as 0 columns

Regarding the comment, it is a minor detail - in particular, using as-color it's impossible to split an ANSI escape across multiple strings.  I can add a note that the behavior is undefined if an ANSI escape is split.