2011-05-21 35 views
2

我編譯Python 2.6.6與谷歌perf工具(tcmalloc)庫,以消除一些與我有的內存問題默認2.6.5。得到2.6.6後,它似乎無法正常工作,因爲我認爲在Ubuntu中安裝默認的2.6.5版本有問題。從軟件渠道安裝的二進制文件(如wxPython和setuptools)都不能在2.6.6中正常運行。這些是否需要重新編譯?任何其他建議,以使其順利運作。我是否仍然可以將2.6.5設置爲默認值而不更改路徑?該路徑首先在usr/local/bin中查找。編譯Python 2.6.6和需要外部軟件包wxPython,setuptools等...在Ubuntu

回答

5

一個很好的通用經驗法則是從不使用使用默認系統安裝的Python,用於超出各種系統管理腳本的任何軟件開發。這適用於包括Linux和OS/X在內的所有UNIX。

取而代之的是,用您需要的庫(Python和C)構建一個您可以控制的良好Python發行版,並將此tarball安裝在非系統目錄中,例如/ opt/devpy或/ data/package/python或/ home/python。爲什麼在2.7.2可用的時候遇到2.6?

當您構建它時,請確保它的所有依賴項都位於其自己的目錄樹(RPATH)中,並且任何系統依賴項(.so文件)都被複制到其目錄樹中。這是我的版本。如果你只是運行整個shell腳本,它可能不起作用。我總是將這些部分複製並粘貼到終端窗口中,並驗證每一步都可以正常工作。確保您的終端屬性設置爲允許大量的回捲行,或者一次只粘貼幾行。

(實際上,做了一些調整,我認爲這可能是可運行的腳本,但我會建議像./pybuild.sh >pylog 2>&1這樣你就可以通過輸出梳理和確認一切建行之後。

這個建在Ubuntu 64位

#!/bin/bash 

shopt -s compat40 

export WGET=echo 
#uncomment the following if you are running for the first time 
export WGET=wget 

sudo apt-get -y install build-essential 
sudo apt-get -y install zlib1g-dev libxml2-dev libxslt1-dev libssl-dev libncurses5-dev 
sudo apt-get -y install libreadline6-dev autotools-dev autoconf automake libtool 
sudo apt-get -y install libsvn-dev mercurial subversion git-core 
sudo apt-get -y install libbz2-dev libgdbm-dev sqlite3 libsqlite3-dev 
sudo apt-get -y install curl libcurl4-gnutls-dev 
sudo apt-get -y install libevent-dev libev-dev librrd4 rrdtool 
sudo apt-get -y install uuid-dev libdb4.8-dev memcached libmemcached-dev 
sudo apt-get -y install libmysqlclient-dev libexpat1-dev 

cd ~ 
$WGET 'http://code.google.com/p/google-perftools/downloads/detail?name=google-perftools-1.7.tar.gz' 
$WGET http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz 
tar zxvf Python-2.7.2.tgz 
cd Python-2.7.2 

#following is needed if you have an old version of Mercurial installed 
#export HAS_HG=not-found 

# To provide a uniform build environment 
unset PYTHONPATH PYTHONSTARTUP PYTHONHOME PYTHONCASEOK PYTHONIOENCODING 
unset LD_RUN_PATH LD_LIBRARY_PATH LD_DEBUG LD_TRACE_LOADED_OBJECTS 
unset LD_PRELOAD SHLIB_PATH LD_BIND_NOW LD_VERBOSE 

## figure out whether this is a 32 bit or 64 bit system 
m=`uname -m` 
if [[ $m =~ .*64 ]]; then 
    export CC="gcc -m64" 
    NBITS=64 
elif [[ $m =~ .*86 ]]; then 
    export CC="gcc -m32" 
    NBITS=32 
else # we are confused so bail out 
    echo $m 
    exit 1 
fi 

# some stuff related to distro independent build 
# extra_link_args = ['-Wl,-R/data1/python27/lib'] 
#--enable-shared and a relative 
# RPATH[0] (eg LD_RUN_PATH='${ORIGIN}/../lib') 

export TARG=/data1/packages/python272 
export TCMALLOC_SKIP_SBRK=true 
#export CFLAGS='-ltcmalloc' # Google's fast malloc 
export COMMONLDFLAGS='-Wl,-rpath,\$$ORIGIN/../lib -Wl,-rpath-link,\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib -Wl,-z,origin -Wl,--enable-new-dtags' 
# -Wl,-dynamic-linker,$TARG/lib/ld-linux-x86-64.so.2 
export LDFLAGS=$COMMONLDFLAGS 
./configure --prefix=$TARG --with-dbmliborder=bdb:gdbm --enable-shared --enable-ipv6 

# if you have ia32-libs installed on a 64-bit system 
#export COMMONLDFLAGS="-L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,$TARG/lib32 -Wl,-rpath,$TARG/usr/lib32" 

make 
# ignore failure to build the following since they are obsolete or deprecated 
# _tkinter bsddb185 dl imageop sunaudiodev 

#install it and collect any dependency libraries - not needed with RPATH 
sudo mkdir -p $TARG 
sudo chown `whoami`.users $TARG 
make install 

# collect binary libraries ##REDO THIS IF YOU ADD ANY ADDITIONAL MODULES## 
function collect_binary_libs { 
cd $TARG 
find . -name '*.so' | sed 's/^/ldd -v /' >elffiles 
echo "ldd -v bin/python" >>elffiles 
chmod +x elffiles 
./elffiles | sed 's/.*=> //;s/ .*//;/:$/d;s/^    *//' | sort -u | sed 's/.*/cp -L & lib/' >lddinfo 
# mkdir lib 
chmod +x lddinfo 
./lddinfo 
cd ~ 
} 
collect_binary_libs 

#set the path 
cd ~ 
export PATH=$TARG/bin:$PATH 

#installed setuptools 
$WGET http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg 
chmod +x setuptools-0.6c11-py2.7.egg 
./setuptools-0.6c11-py2.7.egg 

#installed virtualenv 
tar zxvf virtualenv-1.6.1.tar.gz 
cd virtualenv-1.6.1 
python setup.py install 
cd ~ 

# created a base virtualenv that should work for almost all projects 
# we make it relocatable in case its location in the filesystem changes. 
cd ~ 
python virtualenv-1.6.1/virtualenv.py /data1/py27base # first make it 
python virtualenv-1.6.1/virtualenv.py --relocatable /data1/py27base #then relocatabilize 

# check it out 
source ~/junk/bin/activate 
python --version 

# fill the virtualenv with useful modules 
# watch out for binary builds that may have dependency problems 

export LD_RUN_PATH='\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib' 
easy_install pip 
pip install cython 
pip install lxml 
pip install httplib2 
pip install python-memcached 
pip install amqplib 
pip install kombu 
pip install carrot 
pip install py_eventsocket 
pip install haigha 

# extra escaping of $ signs 
export LDFLAGS='-Wl,-rpath,\$\$$ORIGIN/../lib:\$\$$ORIGIN/../../lib -Wl,-rpath-link,\$\$$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags' 
# even more complex to build this one since we need some autotools and 
# have to pull source from a repository 
mkdir rabbitc 
cd rabbitc 
hg clone http://hg.rabbitmq.com/rabbitmq-codegen/ 
hg clone http://hg.rabbitmq.com/rabbitmq-c/ 
cd rabbitmq-c 
autoreconf -i 
make clean 
./configure --prefix=/usr 
make 
sudo make install 
cd ~ 

# for zeromq we get the latest source of the library 
$WGET http://download.zeromq.org/zeromq-2.1.7.tar.gz 
tar zxvf zeromq-2.1.7.tar.gz 
cd zeromq-2.1.7 
make clean 
./configure --prefix=/usr 
make 
sudo make install 
cd ~ 
# need less escaping of $ signs 
export LDFLAGS='-Wl,-rpath,\$ORIGIN/../lib:\$ORIGIN/../../lib -Wl,-rpath-link,\$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags' 
pip install pyzmq 
pip install pylibrabbitmq # need to build C library and install first 
pip install pylibmc 
pip install pycurl 
export LDFLAGS=$COMMONLDFLAGS 


pip install cherrypy 
pip install pyopenssl # might need some ldflags on this one? 
pip install diesel 
pip install eventlet 
pip install fapws3 
pip install gevent 
pip install boto 
pip install jinja2 
pip install mako 
pip install paste 
pip install twisted 
pip install flup 
pip install pika 
pip install pymysql 
# pip install py-rrdtool # not on 64 bit??? 
pip install PyRRD 
pip install tornado 
pip install redis 

# for tokyocabinet we need the latest source of the library 
$WGET http://fallabs.com/tokyocabinet/tokyocabinet-1.4.47.tar.gz 
tar zxvf tokyocabinet-1.4.47.tar.gz 
cd tokyocabinet-1.4.47 
make clean 
./configure --prefix=/usr --enable-devel 
make 
sudo make install 
cd .. 
$WGET http://fallabs.com/tokyotyrant/tokyotyrant-1.1.41.tar.gz 
tar zxvf tokyotyrant-1.1.41.tar.gz 
cd tokyotyrant-1.1.41 
make clean 
./configure --prefix=/usr --enable-devel 
make 
sudo make install 
cd .. 
pip install tokyo-python 
pip install solrpy 
pip install pysolr 
pip install sunburnt 
pip install txamqp 
pip install littlechef 
pip install PyChef 
pip install pyvb 
pip install bottle 
pip install werkzeug 
pip install BeautifulSoup 
pip install XSLTools 
pip install numpy 
pip install coverage 
pip install pylint 
# pip install PyChecker ??? 
pip install pycallgraph 
pip install mkcode 
pip install pydot 
pip install sqlalchemy 
pip install buzhug 
pip install flask 
pip install restez 
pip install pytz 
pip install mcdict 
# need less escaping of $ signs 

pip install py-interface 
# pip install paramiko # pulled in by another module 
pip install pexpect 

# SVN interface 
$WGET http://pysvn.barrys-emacs.org/source_kits/pysvn-1.7.5.tar.gz 
tar zxvf pysvn-1.7.5.tar.gz 
cd pysvn-1.7.5/Source 
python setup.py backport 
python setup.py configure 
make 
cd ../Tests 
make 
cd ../Sources 
mkdir -p $TARG/lib/python2.7/site-packages/pysvn 
cp pysvn/__init__.py $TARG/lib/python2.7/site-packages/pysvn 
cp pysvn/_pysvn_2_7.so $TARG/lib/python2.7/site-packages/pysvn 
cd ~ 

# pip install protobuf #we have to do this the hard way 
$WGET http://protobuf.googlecode.com/files/protobuf-2.4.1.zip 
unzip protobuf-2.4.1.zip 
cd protobuf-2.4.1 
make clean 
./configure --prefix=/usr 
make 
sudo make install 
cd python 
python setup.py install 
cd ~ 

pip install riak 
pip install ptrace 
pip install html5lib 
pip install metrics 

#redo the "install binary libraries" step 
collect_binary_libs 

# link binaries in the lib directory to avoid search path errors and also 
# to reduce the number of false starts to find the library 
for i in `ls $TARG/lib/python2.7/lib-dynload/*.so` 
do 
ln -f $i $TARG/lib/`basename $i` 
done 
# for the same reason link the whole lib directory to some other places in the tree 
ln -s ../.. $TARG/lib/python2.7/site-packages/lib 

# bundle it up and save it for packaging 
cd/
tar cvf - .$TARG |gzip >~/py272-$NBITS.tar.gz 
cd ~ 

# after untarring on another machine, we have a program call imports.py which imports 
# every library as a quick check that it works. For a more positive check, run it like this 
# strace -e trace=stat,fstat,open python imports.py >strace.txt 2>&1 
# grep -v ' = -1' strace.txt |grep 'open(' >opens.txt 
# sed <opens.txt 's/^open("//;s/".*//' |sort -u |grep -v 'dynload' |grep '\.so' >straced.txt 
# ls -1d /data1/packages/python272/lib/* |sort -u >lib.txt 
# then examine the strace output to see how many places it searches before finding it. 
# a successful library load will be a call to open that doesn't end with ' = -1' 
# If it takes too many tries to find a particular library, then another symbolic link may 
# be a good idea 
+0

我會盡快給你解決。我明白爲什麼現在永遠不要這麼做。很好的安裝腳本。 – 2011-07-08 05:40:07

+0

你不會有任何機會有這個Python 3的更新版本,是嗎?我正在調整你的3.3,但有些部分比其他部分調整,部分給我的困難。 (例如,Python 3.3現在使用wheel文件格式,即'.whl') – 2014-01-08 02:51:52

+0

@BenK是否更新了分發版本,以便獲得最新的pip命令?這對於.whl文件是必要的。爲了將其移植到Python 3,最重要的事情是捕獲構建過程的日誌,尋找錯誤,然後使用strace來確定實際加載的動態庫文件。有些軟件包需要一些幫助才能將這些文件複製到正確的目錄中。 – 2014-01-09 18:32:42

-1

如果您編譯2.6.6並從repos安裝2.6.5,那麼ubuntu在找到您正在使用的python時會發生衝突。

我將此舉標記爲超級用戶。

+0

該路徑首先指向usr/local/bin,因此它找到2.6.6。在像ipython這樣指向2.6.5的應用程序中,它沒有問題。一些應用程序使用安裝的默認python運行。像rabbitvcs和我在那裏看到的問題。這對我來說很簡單。主要是我必須爲2.6.6編譯所有的包,比如setuptools和wxPython,以處理這些包。他們是否可以不使用軟件倉庫中的軟件包? – 2011-05-22 02:42:56

+0

我建議刪除2.6.5,ubuntu已經搞砸了很多軟件包,python可能就是其中之一。 – 2011-05-22 02:49:20

+0

我知道它工作正常。我只是修改pythonpath來查看默認的dist-packages。如果其他人需要幫助,只需輸出PYTHONPATH = $ PYTHONPATH:/usr/lib/python2.6/dist-packages。/ usr/local中的python不會自動查找像默認2.6.5那樣的dist-packages。不幸的是,我仍然遇到內存問題。 – 2011-05-22 03:32:33

0

我敢肯定,你必須編譯wxPython的來要與使用Python的版本。這一直是與其他人誰做這樣的事情的情況下wxPython郵件列表,我認爲這適用於大多數包,特別是如果他們有任何C/C++組件,就像wxPython一樣。根據我的經驗,純Python包有時可以從一個版本轉移到另一個版本。

有相當廣泛的wxPython構建說明在這裏:如果您遇到任何問題http://wxpython.org/BUILD-2.8.html

羅賓·鄧恩等人的wxPython的郵件列表上是非常有益的。

相關問題