Python-多处理意外结果
作者:互联网
我有一些包含迭代器的代码,效果很好:
import multiprocessing
m = [0,1,2,3]
class gener(object):
def __init__(self, m):
self.m = m
self.c = 0
def __iter__(self):
return self
def next(self):
time.sleep(1)
ret = self.m[self.c]
self.c += 1
return ret
tt = gener(m)
def gen(t):
return t.next()
print gen(tt)
print gen(tt)
print gen(tt)
出:
06001
但是,如果我尝试将其插入并行进程中,则不会得到预期的结果:
import time
import multiprocessing
m = [0,1,2,3]
class gener(object):
def __init__(self, m):
self.m = m
self.c = 0
def __iter__(self):
return self
def next(self):
time.sleep(1)
ret = self.m[self.c]
self.c += 1
return ret
tt = gener(m)
def gen(t):
return t.next()
job1 = multiprocessing.Process(target=gen, args=(tt,))
print job1.start()
job2 = multiprocessing.Process(target=gen, args=(tt,))
print job2.start()
job3 = multiprocessing.Process(target=gen, args=(tt,))
print job3.start()
出:
06003
我不知道如何通过并行使用此迭代器.
有谁能够帮助我?
谢谢!
更新:
跟随@Anand S Kumar非常有用的帮助,我更新了我的代码,并且工作正常,除了输出是模棱两可的,目前,我正在尝试找出问题所在,也许这将是另一个线程的问题,也许是Anand会帮助我:)):
from threading import Thread, Lock
import time
m = [0,1,2,3]
starter = 0
class gener(object):
def __init__(self, m):
self.m = m
self.c = 0
def __iter__(self):
return self
def next(self):
time.sleep(1)
ret = self.m[self.c]
self.c += 1
return ret
tt = gener(m)
def f(t):
global starter
lock = Lock()
lock.acquire()
try:
starter = t.next()
finally:
lock.release()
t1 = Thread(target=f,args=(tt,))
t1.start()
t2 = Thread(target=f,args=(tt,))
t2.start()
t3 = Thread(target=f,args=(tt,))
t3.start()
t1.join()
print starter
t2.join()
print starter
t3.join()
print starter
具有相同代码的不同输出:
06005
解决方法:
您正在尝试打印job.start()函数的返回值,该函数不返回任何内容,因此将显示None.
您可以将print语句移到gen(t)函数中,而不是打印job.start()的返回值,例如-
def gen(t):
print t.next()
然后运行程序,而不打印job.start().
如果要从该函数接收返回值,则可以使用多处理模块中的Pool. [Documentation]
文档中的一个示例-
from multiprocessing import Pool
def f(x):
return x*x
if __name__ == '__main__':
pool = Pool(processes=4) # start 4 worker processes
result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously
print result.get(timeout=1) # prints "100" unless your computer is *very* slow
print pool.map(f, range(10))
但是请注意,您实际上是在创建多个进程,而不是线程,它们不会共享全局变量.
我相信您想要的是线程,也许像下面的示例将帮助您入门-
from threading import Thread, Lock
m = [0,1,2,3]
starter = 0
class gener(object):
def __init__(self, m):
self.m = m
self.c = 0
def __iter__(self):
return self
def next(self):
ret = self.m[self.c]
self.c += 1
return ret
tt = gener(m)
def f(t):
global starter
lock = Lock()
lock.acquire()
try:
starter = t.next()
finally:
lock.release()
t1 = Thread(target=f,args=(tt,))
t1.start()
t2 = Thread(target=f,args=(tt,))
t2.start()
t1.join()
t2.join()
标签:parallel-processing,python 来源: https://codeday.me/bug/20191120/2041260.html