摘要:和類是高級類,大部分情況下只要學會使用即可,無需關注其實現細節。類與類十分相似,只不過一個是處理進程,一個是處理線程,可根據實際需要選擇。示例運行結果不同機器運行結果可能不同。
concurrent.futures模塊
該模塊主要特色在于ThreadPoolExecutor 和 ProcessPoolExecutor 類,這兩個類都繼承自concurrent.futures._base.Executor類,它們實現的接口能分別在不同的線程或進程中執行可調用的對象,它們都在內部維護著一個工作線程或者進程池。
ThreadPoolExecutor 和 ProcessPoolExecutor 類是高級類,大部分情況下只要學會使用即可,無需關注其實現細節。
####ProcessPoolExecutor 類
>class ThreadPoolExecutor(concurrent.futures._base.Executor) >| This is an abstract base class for concrete asynchronous executors. >| Method resolution order: >| ThreadPoolExecutor | concurrent.futures._base.Executor | builtins.object | | Methods defined here: | | init(self, max_workers=None, thread_name_prefix="") | Initializes a new ThreadPoolExecutor instance. | | Args: | max_workers: The maximum number of threads that can be used to | execute the given calls. | thread_name_prefix: An optional name prefix to give our threads. | | shutdown(self, wait=True) | Clean-up the resources associated with the Executor. | | It is safe to call this method several times. Otherwise, no other | methods can be called after this one. | | Args: | wait: If True then shutdown will not return until all running | futures have finished executing and the resources used by the | executor have been reclaimed. | | submit(self, fn, *args, **kwargs) | Submits a callable to be executed with the given arguments. | | Schedules the callable to be executed as fn(*args, **kwargs) and returns | a Future instance representing the execution of the callable. | | Returns: | A Future representing the given call. | | ---------------------------------------------------------------------- | Methods inherited from concurrent.futures._base.Executor: | | enter(self) | | exit(self, exc_type, exc_val, exc_tb) | | map(self, fn, *iterables, timeout=None, chunksize=1) | Returns an iterator equivalent to map(fn, iter). | | Args: | fn: A callable that will take as many arguments as there are | passed iterables. | timeout: The maximum number of seconds to wait. If None, then there | is no limit on the wait time. | chunksize: The size of the chunks the iterable will be broken into | before being passed to a child process. This argument is only | used by ProcessPoolExecutor; it is ignored by | ThreadPoolExecutor. | | Returns: | An iterator equivalent to: map(func, *iterables) but the calls may | be evaluated out-of-order. | | Raises: | TimeoutError: If the entire result iterator could not be generated | before the given timeout. | Exception: If fn(*args) raises for any values.
初始化可以指定一個最大進程數作為其參數 max_workers 的值,該值一般無需指定,默認為當前運行機器的核心數,可以由os.cpu_count()獲??;類中含有方法:
map()方法,與python內置方法map() 功能類似,也就是映射,參數為:
一個可調用函數 fn
一個迭代器 iterables
超時時長 timeout
塊數chuncksize 如果大于1, 迭代器會被分塊處理
---->> 該函數有一個特性:其返回結果與調用開始的順序是一致的;在調用過程中不會產生阻塞,也就是說可能前者被調用執行結束之前,后者被調用已經執行結束了。
如果一定要獲取到所有結果后再處理,可以選擇采用submit()方法和futures.as_completed函數結合使用。
shutdown()方法,清理所有與當前執行器(executor)相關的資源
submit() 方法,提交一個可調用對象給fn使用
從concurrent.futures._base.Executor繼承了__enter__() 和 __exit__()方法,這意味著ProcessPoolExecutor 對象可以用于with 語句。
from concurrent import futures with futures.ProcessPoolExecutor(max_works=3) as executor: executor.map()ThreadPoolExecutor類
class ThreadPoolExecutor(concurrent.futures._base.Executor) | This is an abstract base class for concrete asynchronous executors. | | Method resolution order: | ThreadPoolExecutor | concurrent.futures._base.Executor | builtins.object | | Methods defined here: | | init(self, max_workers=None, thread_name_prefix="") | Initializes a new ThreadPoolExecutor instance. | | Args: | max_workers: The maximum number of threads that can be used to | execute the given calls. | thread_name_prefix: An optional name prefix to give our threads. | | shutdown(self, wait=True) | Clean-up the resources associated with the Executor. | | It is safe to call this method several times. Otherwise, no other | methods can be called after this one. | | Args: | wait: If True then shutdown will not return until all running | futures have finished executing and the resources used by the | executor have been reclaimed. | | submit(self, fn, *args, **kwargs) | Submits a callable to be executed with the given arguments. | | Schedules the callable to be executed as fn(*args, **kwargs) and returns | a Future instance representing the execution of the callable. | | Returns: | A Future representing the given call. | | ---------------------------------------------------------------------- | Methods inherited from concurrent.futures._base.Executor: | | enter(self) | | exit(self, exc_type, exc_val, exc_tb) | | map(self, fn, *iterables, timeout=None, chunksize=1) | Returns an iterator equivalent to map(fn, iter). | | Args: | fn: A callable that will take as many arguments as there are | passed iterables. | timeout: The maximum number of seconds to wait. If None, then there | is no limit on the wait time. | chunksize: The size of the chunks the iterable will be broken into | before being passed to a child process. This argument is only | used by ProcessPoolExecutor; it is ignored by | ThreadPoolExecutor. | | Returns: | An iterator equivalent to: map(func, *iterables) but the calls may | be evaluated out-of-order. | | Raises: | TimeoutError: If the entire result iterator could not be generated | before the given timeout. | Exception: If fn(*args) raises for any values.
與ProcessPoolExecutor 類十分相似,只不過一個是處理進程,一個是處理線程,可根據實際需要選擇。
示例from time import sleep, strftime from concurrent import futures def display(*args): print(strftime("[%H:%M:%S]"), end="") print(*args) def loiter(n): msg = "{}loiter({}): doing nothing for {}s" display(msg.format(" "*n, n, n)) sleep(n) msg = "{}loiter({}): done." display(msg.format(" "*n, n)) return n*10 def main(): display("Script starting") executor = futures.ThreadPoolExecutor(max_workers=3) results = executor.map(loiter, range(5)) display("results:", results) display("Waiting for individual results:") for i, result in enumerate(results): display("result {} : {}".format(i, result)) if __name__ == "__main__": main()
運行結果:
[20:32:12]Script starting [20:32:12]loiter(0): doing nothing for 0s [20:32:12]loiter(0): done. [20:32:12] loiter(1): doing nothing for 1s [20:32:12] loiter(2): doing nothing for 2s [20:32:12]results:.result_iterator at 0x00000246DB21BC50> [20:32:12]Waiting for individual results: [20:32:12] loiter(3): doing nothing for 3s [20:32:12]result 0 : 0 [20:32:13] loiter(1): done. [20:32:13] loiter(4): doing nothing for 4s [20:32:13]result 1 : 10 [20:32:14] loiter(2): done. [20:32:14]result 2 : 20 [20:32:15] loiter(3): done. [20:32:15]result 3 : 30 [20:32:17] loiter(4): done. [20:32:17]result 4 : 40
不同機器運行結果可能不同。
示例中設置max_workers=3,所以代碼一開始運行,則有三個對象(0,1,2)被執行loiter() 操作; 三秒后,對象0運行結束,得到結果result 0之后,對象3才開始被執行,同理,對象4的執行時間在對象1執行結果result 1打印結束之后。
文章版權歸作者所有,未經允許請勿轉載,若此文章存在違規行為,您可以聯系管理員刪除。
轉載請注明本文地址:http://specialneedsforspecialkids.com/yun/42326.html
摘要:標準庫中所有阻塞型函數都會釋放,允許其他線程運行。如果調用引發異常,那么當從迭代器檢索其值時,將引發異常??偨Y自版就支持線程了,只不過是使用線程的最新方式。類封裝了模塊的組件,使使用線程變得更加方便。下一篇筆記應該是使用處理并發。 作為Python程序員,平時很少使用并發編程,偶爾使用也只需要派生出一批獨立的線程,然后放到隊列中,批量執行。所以,不夸張的說,雖然我知道線程、進程、并行、...
摘要:本文重點掌握異步編程的相關概念了解期物的概念意義和使用方法了解中的阻塞型函數釋放的特點。一異步編程相關概念阻塞程序未得到所需計算資源時被掛起的狀態。 導語:本文章記錄了本人在學習Python基礎之控制流程篇的重點知識及個人心得,打算入門Python的朋友們可以來一起學習并交流。 本文重點: 1、掌握異步編程的相關概念;2、了解期物future的概念、意義和使用方法;3、了解Python...
摘要:正文總所周知,和根本就是兩個東西,每次因為這個兼容性的問題都會把自己搞瘋。提供了模塊,把下一個新版本的特性導入到當前版本,于是我們就可以在當前版本中測試一些新版本的特性。傳送門不多,才個。 寫在前面 我是在學習cs231n的assignment3的課程,發現里面的代碼大量頻繁出現了這個庫,那我就很奇怪了,為什么有個future這個奇怪名字的庫會出現呢?到底這個庫又有什么用?下面就讓我為...
摘要:圖片下載屬于操作,比較耗時,基于此,可以利用中的多線程將其實現。更多精彩滾雪球學完結滾雪球學第二輪完結滾雪球學第三輪滾雪球學番外篇完結 在 python 編碼過程中...
閱讀 1398·2021-11-22 15:11
閱讀 2842·2019-08-30 14:16
閱讀 2760·2019-08-29 15:21
閱讀 2918·2019-08-29 15:11
閱讀 2459·2019-08-29 13:19
閱讀 2988·2019-08-29 12:25
閱讀 422·2019-08-29 12:21
閱讀 2836·2019-08-29 11:03