bug#48902: 28.0.50; Directory names containing apostrophes and backticks cause problems

[Alan Third]

On Tue, Jun 08, 2021 at 05:02:25PM +0300, Eli Zaretskii wrote:
> > Date: Tue, 8 Jun 2021 14:00:17 +0100
> > From: Alan Third <alan@idiocy.org>
> > Cc: larsi@gnus.org, naofumi@yasufuku.dev, 48902@debbugs.gnu.org,
> >     salutis@me.com
> > 
> > > It looks like stringWithLispString encodes into UTF-16?  But file
> > > names on macOS should be encoded in UTF-8, and in fact
> > > allocInitFromFile already does TRT when it calls ENCODE_FILE, just
> > > before stringWithLispString is called.  So I think the patch is
> > > correct.
> > > 
> > > (UTF-16 encoding on macOS is for ENCODE_SYSTEM, right?)
> > 
> > I think you're right. But confusingly initByReferencingFile takes an
> > NSString which is a UTF-16 format string, so if I remove all the calls
> > to ENCODE_FILE, stringWithLispString works fine.
> > 
> > I guess we just need to make a note that stringWithLispString cannot
> > handle UTF-8 encoded filenames, unless someone has a smarter solution.
> 
> If you do need a UTF-16 encoded string, then instead of ENCODE_FILE
> you can call code_convert_string_norecord with Qutf_16.  There's no
> need to invent or use a private UTF-16 encoder there, and you also get
> rid of an unnecessary extra UTF-8 encoding as a bonus.

In this case the call to ENCODE_FILE in allocInitFromFile is actually
redundant because image_find_image_fd already calls ENCODE_FILE on the
filename before passing it back. So we get a UTF-8 string no matter
what.

NSString can read in almost anything, and Mattias extended it to read
in multibyte (and ascii) lisp strings, so we don't need a UTF-16 input
specifically. It would probably be nice if NSString was also able to
recognise that a lisp string is UTF-8 and handle that itself, but I
don't think that's really possible, unless we make the assumption that
any unibyte string it's passed will already be ascii or UTF-8.

I don't know if that's a reasonable assumption.

-- 
Alan Third