top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

apply Pool map with a method of a class and a list in pythob

+2 votes
541 views

I would like to apply the Pool.map method to a member of a class. Here is a small example that shows what I would like to do:

from multiprocessing import Pool

class A(object):
 def __init__(self,x):
 self.value = x
 def fun(self,x):
 return self.value**x

l = range(10)

p = Pool(4)

op = p.map(A.fun,l)

using this with the normal map doesn't cause any problem

This fails because it says that the methods can't be pickled. (I assume it has something to do with the note in the documentation: "functionality within this package requires that the __main__ module be importable by the children.", which is obscure to me).

I would like to understand two things: why my code fails and when I can expect it to fail? what is a possible workaround?

posted Aug 6, 2013 by Deepak Dasgupta

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

2 Answers

0 votes

Do you ever instantiate any A() objects? You're attempting to call an unbound method without passing it a 'self'.

You may find the results completely different in Python 2 vs Python 3, and between bound and unbound methods. In Python 3, an unbound method is simply a function. In both versions, a bound method carries its
first argument around, so it has to be something different. Play around with it a bit.

answer Aug 6, 2013 by Salil Agrawal
0 votes

I have tried a lot of variations, instantiating the object, creating lambda functions that use the unbound version of fun (A.fun.__func__) etc etc.. I have played around it quite a bit before posting.

As far as I have understood the problem is due to the fact that Pool pickle the function and copy it in the various pools.. But since the methods cannot be pickled this fails..

The same example I posted won't run in Python 3.2 neither (I am mostly interested in a solution for Python 2.7, sorry I forgot to mention that).

answer Aug 6, 2013 by anonymous
Similar Questions
+3 votes
Python: Is there way to get list of instances of particular class through class itself?

Is there way to get list of instances of particular class through class itself? via metaclass or any other method?

If class is object is it possible to delete it? If it is possible then how instances of that class will behave?

+1 vote
Define a class containing methods for reading a file and then store lines in external variable

I want to build a class that perform various functions to the content(or lines) of any given file. I want to first include the opening and reading file function into that class, then add other methods.

Here is what I wrote for opening file and reading the lines:

class FileOpration:
 def __init__(self,name):
 self.name = name
 self.filename = self.name
 def readAllline(self):
 open(self.name).readlines()

file1 = FileOpration("myfile.txt")
print file1.filename
allines = file1.readAllline()
print allines

I define self.name variable because I want to store the specific file that I read. This works fine by the test code of print file1.filename. I saw "myfile.txt" on the terminal output.

However, the readAllline() method dose not work. The code only give me "None" as the output on terminal

+2 votes
Tomcat Connection Pool Alias

With Tomcat 7.0.40 version, we are using Tomcat DBCP for pooling connections.

We are able to pool connections for different aliases. But for one alias, connections are not pooling. The user and password is good because the same alias in apache dbcp works fine.

Why are we not able to pool any connections here?

0 votes
Class hierarchy problem in Python

I have a hierarchy of classes (SubA, SubAB, SubB, ..., SubBCA, SubC,...), each of which is inheriting from a chain of superclasses with a common baseclass(Sup) on top. (So far, no problem)

Now, I want to create instances of the correct subclasstype as decided by the common baseclass, like this:

i = Sup(args_allowing_the_baseclass_to_deduce_correct_subclass)

where i can be of any class except Sup itself (as decided by Sup)

Now, the problem:

How to design the __new__() and __init__() methods for the various classes in order to achieve what I want?

...