Re: [bug-gawk] [External] Re: Invalid Characters Causing Problems in awk 4.0.2

[arnold]

Gilbert,

Hi. Welcome to the wonderful world of Unicode, UTF-8, and multibyte encodings.
It is something of a sudden dunk into the cold, deep end of the lake if you've
never been exposed to it before.

Gawk works on characters, not bytes, based on the current locale as
provided by various LC_* environment variables.  This has been the case
since about version 3.1.5.

You have a few options to cause gawk to treat each byte as its own
character:

1. Put LC_ALL=C into the environment. That should override the other
environment variables.

2. Use gawk's -b option.

Or you can adjust your scripts to understand that bytes != characters.

In any case, I would recommend using version 4.2.1, which is the current
released version, instead of 4.0.2 which is 6 years old.

Best of luck,

Arnold