bug#40555: 27.0.90; out of bound array access in setup_process_coding_systems

[Matthieu Hauglustaine]

Apologies for the duplicate.

I can confirm I reproduced with a built on branch-27, and applying Robert's patch fixes the issue.

Thanks,

Matthieu

On Sat, Apr 11, 2020 at 6:05 PM Eli Zaretskii <eliz@gnu.org> wrote:

merge 40555 40023
thanks

> From: Matthieu Hauglustaine <matt.hauglustaine@gmail.com>
> Date: Sat, 11 Apr 2020 17:24:16 +0200
>
> I've experienced a EXC_BAD_ACCESS when using Emacs 27.0.90 on OS X
> 10.15.
>
> The root cause appears to be an out of bound access on
> proc_decode_coding_system (src/process.c:7988), in
> setup_process_coding_systems() when calling setup_coding_system(). This
> results in an invalid write to coding->id from
> CHECK_CODING_SYSTEM_GET_ID (src/coding.c:5678). [1] for the stacktrace.
>
> On Emacs initialization (init_process_emacs(), src/emacs.c:8234),
> RLIMIT_NOFILE.rlim_cur is set to FD_SETSIZE, and the assumption seem to
> be that this limit will never change for the lifetime of the
> process. proc_decode_coding_system and proc_encode_coding_system are
> declared with a size of FD_SETSIZE (src/process.c:311).
>
> However, on OS X systems, the call to NSURL.getResourceValue:forKey:
> (src/nsfns.c:497), when opening a file, apparently result in a call to
> setrlimit with RLIMIT_NOFILE.rlim_cur > FD_SETSIZE.
>
> Thus, when the number of FDs opened by Emacs is greater than FD_SETSIZE,
> an illegal access is done when make-process is called.

Thankjs, this is bug#40023.  There's a patch there, maybe you could
try it.  If the patch works for you, we could install it on the
emacs-27 branch.