bug#26353: VFS name encoding

[Danny Milosavljevic]

Hi Ludo,

> The problem of how to deal with file name encoding has been discussed on
> the Guile side so hopefully the next release in the 2.2 series will have
> a solution for this.

For what it's worth, I think the sane solution is the Plan 9 solution:  Just 
represent file names as bytevectors.  Programs which don't care about the 
actual name - for example programs that just want to do (for-each unlink 
(scandir (string->utf8 "."))) or something - have no reason to care about the 
encoding at all.  And then use UTF-8 encoding everywhere (for the file names, 
also for everything else) throughout the operating system for the tools that do 
care.

There are also utf8 mount options in the Linux kernel to be able to present 
UTF-8 names to userspace even when the actual names on disk are something else 
- and we should use them.  (I think we should even modify <file-system> flags 
to default to "utf8" or "iocharset=utf8" where possible)

This conversion of UTF-8 to UCS-4 especially is really just busywork.  My 
opinion changed over the years - earlier I was all for UCS-4.  But actually, 
most tools don't care about the actual content of the file names - it's just an 
opaque ID to them (similar to an UUID).  Representing them as something else in 
userspace again (inviting another conversion failure) is just ... unnecessary. 

In any case, it would be different if we had a non-UNIX kernel underneath.  But 
as long as we do have UNIX the kernel VFS interface expects bytevectors, 
preferrably interpreted as UTF-8 (if interpreted at all).

I think this is also the consensus among the major Linux distributions and also 
among lowlevel libraries like glib: They assume one is using UTF-8 filenames 
and default to it whereever possible.