bug#10227: Python installation fails for Python 3

[Stefano Lattarini]

On 11/22/2012 11:35 PM, Roumen Petrov wrote:
> Stefano Lattarini wrote:
>> tags 10227 - moreinfo
>> thanks
>>
>> On 11/21/2012 06:58 PM, Reuben Thomas wrote:
>>> On 21 November 2012 13:41, Stefano Lattarini <address@hidden>wrote:
>>>
>>>> tags 10227 + moreinfo
>>>> thanks
>>>>
>>>> Hi Roumen, Reuben.
>>>>
>>>> I'm going through old open bugs, and I've noticed this one.  Is the
>>>> problem still present, after the recent updates to the python support?
> [SNIP]
>>>> on my Ubuntu 12.10 system). I'm sorry, I can't tell whether, as Roumen
>>>> suggests, this is an Ubuntu packaging bug, or an automake bug.
> It seems to me this is a feature as I found this one:
> 
> http://patch-tracker.debian.org/patch/series/view/python2.6/2.6.6-8/distutils-install-layout.diff
>
The problem didn't lie in the Debian's use of 'dist-packages' instead
of 'site-packages' (as you noticed, that was present in python 2.6
already, and Automake still worked nicely); the issue is that, with
Python 3 on Debian, the call to

    distutils.sysconfig.get_python_lib(0,0,prefix='/usr/local')

returned:

    /usr/local/lib/python3/dist-packages

that is *not* searched by default by the 'import' statement.

>> OK, thanks for explaining it once again.  I can now reproduce the same
>> issue on Debian.  I think this is something we should try to work around,
>> since we cannot have our installation rules broken by default on both
>> Debian and Ubuntu ...
>
> Lets say platforms that use the "/posix_local/" installation scheme
> instead wrong or broken.
>
I failed to parse this, sorry.  Care to rephrase?

>>> It seems to me that this is the sort of thing the pyconfigure people
>>> ought to know about.
>>>
>> By peeking at their repository, I came up with the minimal patch below,
>> which should do the trick.  Can you verify it works for you as well?
>>

>> Signed-off-by: Stefano Lattarini <address@hidden>
>> ---
>>   [SNIP]
>>
>> @@ -122,7 +141,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>        else
>>          am_py_prefix=$prefix
>>        fi
>> -     am_cv_python_pythondir=`$PYTHON -c "import sys; from distutils import 
>> sysconfig; 
>> sys.stdout.write(sysconfig.get_python_lib(0,0,prefix='$am_py_prefix'))" 
>> 2>/dev/null`
>> +     am_cv_python_pythondir=`$PYTHON -c "
>> +$am_python_setup_sysconfig
>> +if can_use_sysconfig:
>> +    sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'})
>> +else:
>> +    from distutils import sysconfig
>> +    sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix')
>> +sys.stdout.write(sitedir)"`
> 
> Tested with python build from master branch on  2012-11-13 - both paths
> (sysconfig and distutils.sysconfig based) return same results.
>
Thanks for pointing that out.  I've given the code a try with a python 3.2
installed from sources, and I can confirm this happens for me as well.  So
no regression for that kind of setup, at least.

On the other hand, for a python 3.2 installed from Debian packages, the
results are different:

    # Code with distutils.sysconfig
    ${prefix}/lib/python3/dist-packages

    # Code with sysconfig
    ${prefix}/lib/python3.2/site-packages

The second result is the correct one; in fact, the 'dist-packages'
directory is meant as the location of python modules installed from
Debian packages (reference: <http://wiki.debian.org/Python>), so we
shouldn't need to install into such a directory; and when ${prefix}
is /usr/local, the "${prefix}/lib/python3.2/site-packages" directory
is searched by default by the 'import' statement.

At this point, I'm convinced my patch is quite correct.

>>        case $am_cv_python_pythondir in
>>        $am_py_prefix*)
>>          am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'`
>> @@ -157,7 +183,14 @@ AC_DEFUN([AM_PATH_PYTHON],
>>        else
>>          am_py_exec_prefix=$exec_prefix
>>        fi
>> -     am_cv_python_pyexecdir=`$PYTHON -c "import sys; from distutils import 
>> sysconfig; 
>> sys.stdout.write(sysconfig.get_python_lib(1,0,prefix='$am_py_exec_prefix'))" 
>> 2>/dev/null`
>> +     am_cv_python_pyexecdir=`$PYTHON -c "
>> +$am_python_setup_sysconfig
>> +if can_use_sysconfig:
>> +    sitedir = sysconfig.get_path('platlib', vars={'base':'$am_py_prefix'})
> 
> Stefano, please use platbase as variable - see Lib/sysconfig.py:
> ....
> INSTALL_SCHEMES = {
>     'posix_prefix': {
> ...
>         'purelib': '{base}/lib/python{py_version_short}/site-packages',
>         'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
> ...
>
Good catch!  I will fix this.

>> +else:
>> +    from distutils import sysconfig
>> +    sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_prefix')
>> +sys.stdout.write(sitedir)"`
>>        case $am_cv_python_pyexecdir in
>>        $am_py_exec_prefix*)
>>          am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'`
> 
> Also I think that python experts should confirm if patch is correct.
>
I'd love that.  Feel free to ping any GNU python expert about giving
a look at this patch.

Thanks,
  Stefano