switching to Python 3.x

[Jonas Hahnfeld]

Hello friends of Python 3!

to make the initial proposal short: With today's patches, I think
'master' would be ready to switch over to Python 3.x.
As a reminder, Python 2 will go EOL on 1st of January 2020 - in 13 days
if I counted correctly. That probably doesn't mean that Python 2.x will
go away immediately, but I strongly believe that active projects should
switch. Eventually Linux distributions will start dropping packages
that depend on Python 2.x, and I personally don't want LilyPond to be
one of them.

There's still one preparatory patch left, but it will break
compatibility with Python 2.4 which is still supported right now:
https://sourceforge.net/p/testlilyissues/issues/5645/
As far as I can tell, the change works fine with Python 2.7. We could
land this immediately before switching to Python 3 if building 'master'
with Python 2.4 is critical. If needed, we would still retain the
ability to bisect via Python 2.7.

The switch itself requires 3 changes which I would propose to squash
into one commit: (I'll put all of this into 
https://sourceforge.net/p/testlilyissues/issues/5646/)
1) Adapt the build system to find and require Python 3.
patch: https://codereview.appspot.com/545370043
2) The largest part of the switch is running 2to3 which is now able to
handle the rest of the conversion automatically. For reference, changes
for current 'master' are here: https://codereview.appspot.com/573340043
3) Fix-up two places in the scripts afterwards:
3a) Remove the call to sys.setdefaultencoding which doesn't exist in
Python 3.x (and is only available right now because of a dirty hack).
3b) Replace cgi.escape by html.escape. While not strictly needed, I
think we should include this change because they removed cgi.escape in
Python 3.8 after only deprecating it in Python 3.2. The replacement
html.escape is only available since that Python 3.2, so we can't do
this before switching.
patch: https://codereview.appspot.com/561270043

One point worth discussing is the future minimum version of Python 3.x.
In the patch, I'm proposing Python 3.5 because it will allow us to
address a few deprecation warnings, especially about module 'imp'.
It should be available in most Linux distributions:
 * CentOS 7 provides Python 3.6 since some minor releases
 * Debian 9 (Stretch) has packages for Python 3.5, Debian 10 (Buster)
even for Python 3.6
 * Ubuntu 16.04 LTS has Python 3.5, version 18.04 LTS has Python 3.6

Debian 8 (Jessie) only has Python 3.4, but even its LTS support will
end next June (2020-06). Would it be acceptable to drop support for
this distribution building 'master'?
On the GUB side, I already added a spec for Python 3.7.4 (also for
Windows via binary packages) and this worked successfully in September.
Is there a major distribution missing that doesn't provide at least
Python 3.5?

Let me know what you think!
Jonas

P.S.: For the curious, I've pushed the mentioned changes as separate
commits to dev/hahnjo/python3. It passes a check against a baseline
created from current 'master' with Python 2, so you might give it a try
on your end ;-)

signature.asc
Description: This is a digitally signed message part