Re: Unicode I/O for GM2

[Benjamin Kowarsch]

On Sun, 24 Mar 2024 at 11:42, Alice Osako wrote:

I am trying to find a suitable way to approach both file and console I/O for my UNICODE library, and I am concerned that - especially regarding console I/O - I may have to write some sort of external C module which would be accessed by FFI in some manner, which would act as a wrapper around the wchar_t functions and handle the transitions to and from the Unicode character types.

This is concerning, as I would rather find a native Modula-2 solution if possible, but in this instance I don't see a way to do so. Most of the existing I/O operations for numbers auto-convert the values into numeric text strings; the only exception to this is the ISO RawIO and SRawIO libraries.

Unfortunately, while I tried to use [S]RawIO to implement ReadUtf8Buffer and WriteUtf8Buffer, it didn't have the desired behavior with console output. I've checked in the two experimental procedures, if anyone is curious, but the salient point is that it doesn't work as intended. In any case, it would have been specific to ISO support.

I am not sure I understand (1) what you are trying to do and (2) what the problem is.

I am assuming though that you have UTF8 input that you want to read into an ARRAY OF UNICHAR in memory, and conversely an ARRAY OF UNICHAR that you want to write out as UTF8 output, where UNICHAR is in UCS-4.

For that, all you need are procedures for reading and writing one or more octets, like procedures ReadOctet, ReadOctets, WriteOctet and WriteOctets in my BasicFileIO library.

https://github.com/m2sf/m2bsk/blob/master/src/lib/IO/BasicFileIO.def

There are several implementations of this library, all conforming to the same interface in

https://github.com/m2sf/m2pp/tree/master/src/imp/BasicFileIO

(1) BasicFileIO.pim.mod -- generic implementation for PIM, using PIM's FileSystem

(2) BasicFileIO.iso.mod -- generic implementation for ISO, using the ISO I/O library

(3) BasicFileIO.posix.mod -- implementation for POSIX, using the C I/O library

(4) BasicFileIO.gpm.mod -- implementation for the Gardens Point Modula-2 compiler

(5) BasicFileIO.ulm.mod -- implementation for the Ulm Modula-2 compiler

The console module is simply a convenience layer on top of Terminal,

https://github.com/m2sf/m2pp/blob/master/src/Terminal.nonpim.def

For Terminal, there are three implementations

(1) for ISO -- https://github.com/m2sf/m2pp/blob/master/src/imp/Terminal.iso.mod

(2) for POSIX -- https://github.com/m2sf/m2pp/blob/master/src/imp/Terminal.posix.mod

Note, the IO library evolved while working on M2PP and some parts of it haven't been backported to M2BSK yet.

I suggest you think about my proposal to move the library out into a separate repo, which would then be a good opportunity to consolidate it. At that point, there would be two possibilities going forward: (1) you simply link to the I/O library from your Unicode project, or (2) we could merge the Unicode library into the I/O library and add another layer on top for reading and writing Unicode text.

regards

benjamin