時(shí)間:2022-05-25 22:12:01 | 來(lái)源:網(wǎng)絡(luò)營(yíng)銷(xiāo)
時(shí)間:2022-05-25 22:12:01 來(lái)源:網(wǎng)絡(luò)營(yíng)銷(xiāo)
選擇了腳本語(yǔ)言就要忍受其速度,這句話在某種程度上說(shuō)明了python作為腳本的一個(gè)不足之處,那就是執(zhí)行效率和性能不夠理想,特別是在performance較差的機(jī)器上,因此有必要進(jìn)行一定的代碼優(yōu)化來(lái)提高程序的執(zhí)行效率。那么我們?cè)撊绾芜M(jìn)行Python性能優(yōu)化呢?接下來(lái)我就在億企邦上跟大家共同探討一下這個(gè)問(wèn)題。本文會(huì)涉及常見(jiàn)的代碼優(yōu)化方法,性能優(yōu)化工具的使用以及如何診斷代碼的性能瓶頸等內(nèi)容,希望可以給Python開(kāi)發(fā)人員一定的參考價(jià)值。O(1) -> O(lg n) -> O(n lg n) -> O(n^2) -> O(n^3) -> O(n^k) -> O(k^n) -> O(n!)因此如果能夠在時(shí)間復(fù)雜度上對(duì)算法進(jìn)行一定的改進(jìn),對(duì)性能的提高不言而喻。但對(duì)具體算法的改進(jìn)不屬于本文討論的范圍,讀者可以自行參考這方面資料。下面的內(nèi)容將集中討論數(shù)據(jù)結(jié)構(gòu)的選擇。
from time import time上述代碼運(yùn)行大概需要16.09seconds。如果去掉行#list = dict.fromkeys(list,True)的注釋?zhuān)瑢ist轉(zhuǎn)換為字典之后再運(yùn)行,時(shí)間大約為8.375 seconds,效率大概提高了一半。因此在需要多數(shù)據(jù)成員進(jìn)行頻繁的查找或者訪問(wèn)的時(shí)候,使用dict而不是list是一個(gè)較好的選擇。
t = time()
list = ['a','b','is','python','jason','hello','hill','with','phone','test',
'dfdf','apple','pddf','ind','basic','none','baecr','var','bana','dd','wrd']
#list = dict.fromkeys(list,True)
print list
filter = []
for i in range (1000000):
for find in ['is','hat','new','list','old','.']:
if find not in list:
filter.append(find)
print "total run time:"
print time()-t
from time import time上述程序的運(yùn)行時(shí)間大概為:
t = time()
lista=[1,2,3,4,5,6,7,8,9,13,34,53,42,44]
listb=[2,4,6,9,23]
intersection=[]
for i in range (1000000):
for a in lista:
for b in listb:
if a == b:
intersection.append(a)
print "total run time:"
print time()-t
total run time:使用set 求交集
38.4070000648
from time import time改為set后程序的運(yùn)行時(shí)間縮減為8.75,提高了4倍多,運(yùn)行時(shí)間大大縮短。讀者可以自行使用表1其他的操作進(jìn)行測(cè)試。
t = time()
lista=[1,2,3,4,5,6,7,8,9,13,34,53,42,44]
listb=[2,4,6,9,23]
intersection=[]
for i in range (1000000):
list(set(lista)&set(listb))
print "total run time:"
print time()-t
from time import time現(xiàn)在進(jìn)行如下優(yōu)化,將長(zhǎng)度計(jì)算提到循環(huán)外,range用xrange代替,同時(shí)將第三層的計(jì)算lista[a]提到循環(huán)的第二層。
t = time()
lista = [1,2,3,4,5,6,7,8,9,10]
listb =[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.01]
for i in range (1000000):
for a in range(len(lista)):
for b in range(len(listb)):
x=lista[a]+listb[b]
print "total run time:"
print time()-t
from time import time上述優(yōu)化后的程序其運(yùn)行時(shí)間縮短為102.171999931。在清單 4 中 lista[a] 被計(jì)算的次數(shù)為1000000*10*10,而在優(yōu)化后的代碼中被計(jì)算的次數(shù)為1000000*10,計(jì)算次數(shù)大幅度縮短,因此性能有所提升。
t = time()
lista = [1,2,3,4,5,6,7,8,9,10]
listb =[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,0.01]
len1=len(lista)
len2=len(listb)
for i in xrange (1000000):
for a in xrange(len1):
temp=lista[a]
for b in xrange(len2):
x=temp+listb[b]
print "total run time:"
print time()-t
from time import time在未進(jìn)行優(yōu)化之前程序的運(yùn)行時(shí)間大概為8.84,如果使用注釋行代替第一個(gè)if,運(yùn)行的時(shí)間大概為6.17。
t = time()
abbreviations = ['cf.', 'e.g.', 'ex.', 'etc.', 'fig.', 'i.e.', 'Mr.', 'vs.']
for i in range (1000000):
for w in ('Mr.', 'Hat', 'is', 'chasing', 'the', 'black', 'cat', '.'):
if w in abbreviations:
#if w[-1] == '.' and w in abbreviations:
pass
print "total run time:"
print time()-t
from time import time同時(shí)要避免:
t = time()
s = ""
list = ['a','b','b','d','e','f','g','h','i','j','k','l','m','n']
for i in range (10000):
for substr in list:
s+= substr
print "total run time:"
print time()-t
s = ""而是要使用:
for x in list:
s += func(x)
slist = [func(elt) for elt in somelist](2)、當(dāng)對(duì)字符串可以使用正則表達(dá)式或者內(nèi)置函數(shù)來(lái)處理的時(shí)候,選擇內(nèi)置函數(shù)。如 str.isalpha(),str.isdigit(),str.startswith(('x', 'yz')),str.endswith(('x', 'yz'))
s = "".join(slist)
out = "<html>%s%s%s%s</html>" % (head, prologue, query, tail)而避免
out = "<html>" + head + prologue + query + tail + "</html>"8、使用列表解析(list comprehension)和生成器表達(dá)式(generator expression)
from time import time使用列表解析:
t = time()
list = ['a','b','is','python','jason','hello','hill','with','phone','test',
'dfdf','apple','pddf','ind','basic','none','baecr','var','bana','dd','wrd']
total=[]
for i in range (1000000):
for w in list:
total.append(w)
print "total run time:"
print time()-t
for i in range (1000000):上述代碼直接運(yùn)行大概需要17s,而改為使用列表解析后 ,運(yùn)行時(shí)間縮短為9.29s。將近提高了一半。生成器表達(dá)式則是在2.4中引入的新內(nèi)容,語(yǔ)法和列表解析類(lèi)似,但是在大數(shù)據(jù)量處理時(shí),生成器表達(dá)式的優(yōu)勢(shì)較為明顯,它并不創(chuàng)建一個(gè)列表,只是返回一個(gè)生成器,因此效率較高。在上述例子上中代碼a = [w for w in list]修改為a = (w for w in list),運(yùn)行時(shí)間進(jìn)一步減少,縮短約為2.98s。
a = [w for w in list]
>>> from timeit import Timer(2)、在循環(huán)的時(shí)候使用 xrange 而不是 range;使用 xrange 可以節(jié)省大量的系統(tǒng)內(nèi)存,因?yàn)?xrange() 在序列中每次調(diào)用只產(chǎn)生一個(gè)整數(shù)元素。而 range() 將直接返回完整的元素列表,用于循環(huán)時(shí)會(huì)有不必要的開(kāi)銷(xiāo)。在 python3 中 xrange 不再存在,里面 range 提供一個(gè)可以遍歷任意長(zhǎng)度的范圍的 iterator。
>>> Timer("t=a;a=b;b=t","a=1;b=2").timeit()
0.25154118749729365
>>> Timer("a,b=b,a","a=1;b=2").timeit()
0.17156677734181258
>>>
import profile程序的運(yùn)行性能分析結(jié)果如下圖所示:
def profileTest():
Total =1;
for i in range(10):
Total=Total*(i+1)
print Total
return Total
if __name__ == "__main__":
profile.run("profileTest()")
import pstats其中sort_stats()方法能夠?qū)ζ史謹(jǐn)?shù)據(jù)進(jìn)行排序,可以接受多個(gè)排序字段,如sort_stats('name', 'file')將首先按照函數(shù)名稱(chēng)進(jìn)行排序,然后再按照文件名進(jìn)行排序。常見(jiàn)的排序字段有calls( 被調(diào)用的次數(shù) ),time(函數(shù)內(nèi)部運(yùn)行時(shí)間),cumulative(運(yùn)行的總時(shí)間)等。此外pstats也提供了命令行交互工具,執(zhí)行python – m pstats后可以通過(guò)help了解更多使用方式。
p = pstats.Stats('testprof')
p.sort_stats("name").print_stats()
C:Documents and SettingsAdministrator>pypy以第5條中的“循環(huán)優(yōu)化后”的循環(huán)為例子,使用python和pypy分別運(yùn)行,得到的運(yùn)行結(jié)果分別如下:
Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 18:31:47)
[PyPy 1.8.0 with MSC v.1500 32 bit] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``PyPy is vast, and contains
multitudes''
>>>>
C:Documents and SettingsAdministrator 桌面 docpython>pypy loop.py可見(jiàn)使用pypy來(lái)編譯和運(yùn)行程序,其效率大大的提高。
total run time:
8.42199993134
C:Documents and SettingsAdministrator 桌面 docpython>python loop.py
total run time:
106.391000032
def sum(int a,int b):在linux上利用gcc編譯為.so文件:
print a+b
[root@v5254085f259 test]# cython sum.pyx
[root@v5254085f259 test]# ls
total 76
4 drwxr-xr-x 2 root root 4096 Apr 17 02:45 .
4 drwxr-xr-x 4 root root 4096 Apr 16 22:20 ..
4 -rw-r--r-- 1 root root 35 Apr 17 02:45 1
60 -rw-r--r-- 1 root root 55169 Apr 17 02:45 sum.c
4 -rw-r--r-- 1 root root 35 Apr 17 02:45 sum.pyx
[root@v5254085f259 test]# gcc -shared -pthread -fPIC -fwrapv -O2B、使用distutils編譯
-Wall -fno-strict-aliasing -I/usr/include/python2.4 -o sum.so sum.c
[root@v5254085f259 test]# ls
total 96
4 drwxr-xr-x 2 root root 4096 Apr 17 02:47 .
4 drwxr-xr-x 4 root root 4096 Apr 16 22:20 ..
4 -rw-r--r-- 1 root root 35 Apr 17 02:45 1
60 -rw-r--r-- 1 root root 55169 Apr 17 02:45 sum.c
4 -rw-r--r-- 1 root root 35 Apr 17 02:45 sum.pyx
20 -rwxr-xr-x 1 root root 20307 Apr 17 02:47 sum.so
from distutils.core import setup編譯完成之后可以導(dǎo)入到 python 中使用:
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("sum", ["sum.pyx"])]
setup(
name = 'sum app',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
[root@v5254085f259 test]# python setup.py build_ext --inplace
running build_ext
cythoning sum.pyx to sum.c
building 'sum' extension
gcc -pthread -fno-strict-aliasing -fPIC -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes -fPIC -I/opt/ActivePython-2.7/include/python2.7
-c sum.c -o build/temp.linux-x86_64-2.7/sum.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/sum.o
-o /root/cpython/test/sum.so
[root@v5254085f259 test]# python億企邦點(diǎn)評(píng):
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 11:24:26)
[GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyximport; pyximport.install()
>>> import sum
>>> sum.sum(1,3)
from time import time測(cè)試結(jié)果:
def test(int n):
cdef int a =0
cdef int i
for i in xrange(n):
a+= i
return a
t = time()
test(10000000)
print "total run time:"
print time()-t
[GCC 4.0.2 20051125 (Red Hat 4.0.2-8)] on linux2Python 測(cè)試代碼
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyximport; pyximport.install()
>>> import ctest
total run time:
0.00714015960693
from time import time從上述對(duì)比可以看到使用Cython的速度提高了將近100多倍。
def test(n):
a =0;
for i in xrange(n):
a+= i
return a
t = time()
test(10000000)
print "total run time:"
print time()-t
[root@v5254085f259 test]# python test.py
total run time:
0.971596002579
關(guān)鍵詞:方法,性能,語(yǔ)言
客戶&案例
營(yíng)銷(xiāo)資訊
關(guān)于我們
客戶&案例
營(yíng)銷(xiāo)資訊
關(guān)于我們
微信公眾號(hào)
版權(quán)所有? 億企邦 1997-2022 保留一切法律許可權(quán)利。