编译Python 3.2和2.7时使用Python using的问题

Question

我正在尝试使用比利斯构建多个版本的python，但我遇到了一些测试失败。这是运行在VM上的：Ubuntu8.04 32位

当我使用--enable-shared选项时会发生这种情况。

build.log文件错误如下所示：

Python 3.2

以下是Python3.2的失败测试：

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -E -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -Wd -E -bb  ./Lib/test/regrtest.py -l 
== CPython 3.2 (r32:88445, Jun 16 2011, 15:07:10) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-3.2/build/test_python_18839
Testing with flags: sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0)

....SNIP....

[197/349] test_os
test test_os failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/test/test_os.py", line 646, in test_exist_ok_existing_directory
    os.makedirs(path, mode=mode, exist_ok=True)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/os.py", line 152, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '@test_18839_tmp/dir1'

....SNIP....

321 tests OK.
1 test failed:
    test_os
27 tests skipped:
    test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
    test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu
    test_dbm_ndbm test_gdb test_ioctl test_kqueue test_ossaudiodev
    test_pep277 test_smtpnet test_socketserver test_sqlite
    test_startfile test_timeout test_tk test_ttk_guionly
    test_unicode_file test_urllib2net test_urllibnet test_winreg
    test_winsound test_xmlrpc_net test_zipfile64
5 skips unexpected on linux2:
    test_dbm_gnu test_dbm_ndbm test_ioctl test_tk test_ttk_guionly
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='ANSI_X3.4-1968'>
make: *** [test] Error 1

Python 2.7.1

以下是Python2.7.1的失败测试：

./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l 
== CPython 2.7.1 (r271:86832, Jun 16 2011, 17:59:28) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/build/test_python_19286

....SNIP....

test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/Lib/test/test_file2k.py", line 219, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

343 tests OK.
2 tests failed:
    test_file test_file2k
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: *** [test] Error 1

Python 2.7

下面是Python2.7的失败测试：

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -E -c 'import sys ; from sysconfig import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -Wd -3 -E -tt ./Lib/test/regrtest.py -l 
== CPython 2.7 (r27:82500, Jun 16 2011, 15:25:09) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7/build/test_python_13541

....SNIP....

test_distutils
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
test test_distutils failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/tests/test_build_ext.py", line 269, in test_get_outputs
    cmd.run()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/ccompiler.py", line 741, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/unixccompiler.py", line 258, in link
    raise LinkError, msg
LinkError: command 'gcc' failed with exit status 1

....SNIP....

test test_file failed -- multiple errors occurred; run in verbose mode for details
test_file2k
test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/test/test_file2k.py", line 211, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

339 tests OK.
3 tests failed:
    test_distutils test_file test_file2k
1 test altered the execution environment:
    test_site
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: [test] Error 1 (ignored)

Answer 1

目前，我设法使用2.7.2完成了以下操作：

首先，安装编译python所需的所有依赖项。

'curl' # not for build, but for steps after
'build-essential',
'libbz2-dev',
'libsqlite3-dev',
'zlib1g-dev',
'libxml2-dev',
'libxslt1-dev',
'libreadline5', # lenny
'libreadline5-dev', # lenny
'libgdbm-dev',
'libgdb-dev',
'libxml2',
'libssl-dev',
'tk-dev',
'libgdbm-dev',
'libexpat1-dev',
'libncursesw5-dev'

( b)现在安装2.7.2：

pythonbrew install --configure="--with-threads --enable-shared" -j2 -v 2.7.2

这将安装，但会被以下错误击中

加载共享库时出现/home/python-deploy/.pythonbrew/pythons/Python-2.7.2/bin/python:错误: libpython2.7.so.1.0:无法打开共享对象文件:没有这样的文件或目录

*为了避免在构建后安装setuptools时出现错误，使用

pythonbrew install --configure="--with-threads--enable-shared" \
                   --force \
                   --no-setuptools \
                   --jobs=2 \
                   --verbose 2.7.2 

( c)最简单的解决方法是(使用正确的pythonbrew安装路径)：

pythonbrew use 2.7.2
export LD_LIBRARY_PATH=$HOME/.pythonbrew/pythons/Python-2.7.2/lib

或者，将pythonbrew路径永久添加到/etc/ld.so.con.d下面的文件中，路径将在启动时添加，或者立即运行ldconfig：

sudo echo /home/user/.pythonbrew/pythons/Python-2.7.2/lib >> /etc/ld.so.conf.d/pythonbrew.conf
sudo ldconfig

( d)安装setuptools或手动分发，即

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py && easy_install pip

你应该准备好走了。只在一个流浪汉上使用2.7.2进行测试。

Answer 2

我想我找到了一种不需要设置LD_LIBRARY_PATH就能完成这个任务的方法。

我发现有一个名为LD_RUN_PATH的环境变量，它与LD_LIBRARY_PATH类似，只是在编译程序时设置它，并且每当运行它时它都会记住该路径(因此您不需要在运行时设置LD_LIBRARY_PATH )。

因此，如果您在运行pythonbrew时适当地设置了LD_RUN_PATH以安装您的Python，那么每当您运行它时，它都会找到自己的共享库。例如，下面的命令为我成功，包括安装distutils和pip，这样模块安装就可以不进行任何进一步的设置了。

LD_RUN_PATH=$HOME/.pythonbrew/pythons/Python-2.7.3/lib pythonbrew install -j2 -C '--enable-shared' --force -v 2.7.3

然后，我可以做pythonbrew use 2.7.3激活它，它只是工作，而不设置任何环境变量在您的外壳，init或任何东西。

当然，您需要预先知道pythonbrew将在哪里安装您的Python，以便您能够正确地设置LD_RUN_PATH。通常情况下，它将是$HOME/.pythonbrew/pythons/Python-$VERSION/lib，但是如果这不起作用，您可以通过实际安装它一次(没有--enable-shared)，然后找到lib目录就可以找到它。

我认为出现这些错误是因为在使用--enable-shared进行安装之后，安装的Python找到了系统Python的共享库并使用了它，如果两个Pythons的构建方式有任何不匹配之处，就会产生问题。使用LD_LIBRARY_PATH或LD_RUN_PATH解决方案都会迫使您使用自己的共享库。

Answer 3

瑞安·汤普森的回答很好. LD_RUN_PATH是正确的治疗方法。

但是，与其硬编码绝对路径，还可以使用：

LD_RUN_PATH='$ORIGIN/../lib'

..。其中$ORIGIN是正在运行的二进制文件的PWD (例如。( python)。

# objdump -x /home/arisinger/.pythonbrew/pythons/Python-2.7.3/bin/python|grep RPATH
  RPATH                $ORIGIN/../lib
# ldd .pythonbrew/pythons/Python-2.7.3/bin/python
    [...]
    libpython2.7.so.1.0 => /home/arisinger/.pythonbrew/pythons/Python-2.7.3/bin/../lib/libpython2.7.so.1.0 (0x00007f43994f6000)
    [...]

..。以上内容还将正确安装pip等。最后，来自man ld.so。

$ORIGIN and rpath
    ld.so  understands the string $ORIGIN (or equivalently ${ORIGIN}) in an
    rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory con‐
    taining  the  application  executable.  Thus, an application located in
    somedir/app could be compiled with gcc  -Wl,-rpath,'$ORIGIN/../lib'  so
    that  it  finds  an  associated shared library in somedir/lib no matter
    where somedir is located in the directory hierarchy.  This  facilitates
    the  creation  of  "turn-key"  applications  that  do  not  need  to be
    installed into special directories, but can instead  be  unpacked  into
    any directory and still find their own shared libraries.

编辑:遗憾的是， venv中断了，因为复制了python二进制文件(更改$ORIGIN)，但没有复制(或符号链接)库，所以venv中的C/内置动态模块最终链接到系统python .不好:-( .

..。IMO这是一个需要修复的bug (在virtualenv中？)，但是一个简单的解决方法是手动地将libpythonX.Y.so*复制/符号链接到venv的lib/ dir中。或者做一些看起来有点令人讨厌的事情(注X.Y.Z)：

LD_RUN_PATH='$ORIGIN/../lib:$ORIGIN/../../../../pythons/Python-X.Y.Z/lib'

..。选择对你来说不那么恶劣的东西；两者都被证实是可行的。