pools - Generic pools of resources

class eventlet.pools.Pool(min_size=0, max_size=4, order_as_stack=False, create=None)

Pool class implements resource limitation and construction.

There are two ways of using Pool: passing a create argument or subclassing. In either case you must provide a way to create the resource.

When using create argument, pass a function with no arguments:

http_pool = pools.Pool(create=httplib2.Http)

If you need to pass arguments, build a nullary function with either lambda expression:

http_pool = pools.Pool(create=lambda: httplib2.Http(timeout=90))

or functools.partial():

from functools import partial
http_pool = pools.Pool(create=partial(httplib2.Http, timeout=90))

When subclassing, define only the create() method to implement the desired resource:

class MyPool(pools.Pool):
    def create(self):
        return MyObject()

If using 2.5 or greater, the item() method acts as a context manager; that’s the best way to use it:

with mypool.item() as thing:
    thing.dostuff()

The maximum size of the pool can be modified at runtime via the resize() method.

Specifying a non-zero min-size argument pre-populates the pool with min_size items. max-size sets a hard limit to the size of the pool – it cannot contain any more items than max_size, and if there are already max_size items ‘checked out’ of the pool, the pool will cause any greenthread calling get() to cooperatively yield until an item is put() in.

create()

Generate a new pool item. In order for the pool to function, either this method must be overriden in a subclass or the pool must be constructed with the create argument. It accepts no arguments and returns a single instance of whatever thing the pool is supposed to contain.

In general, create() is called whenever the pool exceeds its previous high-water mark of concurrently-checked-out-items. In other words, in a new pool with min_size of 0, the very first call to get() will result in a call to create(). If the first caller calls put() before some other caller calls get(), then the first item will be returned, and create() will not be called a second time.

free()

Return the number of free items in the pool. This corresponds to the number of get() calls needed to empty the pool.

get()

Return an item from the pool, when one is available. This may cause the calling greenthread to block.

item()

Get an object out of the pool, for use with with statement.

>>> from eventlet import pools
>>> pool = pools.TokenPool(max_size=4)
>>> with pool.item() as obj:
...     print("got token")
...
got token
>>> pool.free()
4

put(item)

Put an item back into the pool, when done. This may cause the putting greenthread to block.

resize(new_size)

Resize the pool to new_size.

Adjusting this number does not affect existing items checked out of the pool, nor on any greenthreads who are waiting for an item to free up. Some indeterminate number of get()/put() cycles will be necessary before the new maximum size truly matches the actual operation of the pool.

waiting()

Return the number of routines waiting for a pool item.

class eventlet.pools.TokenPool(min_size=0, max_size=4, order_as_stack=False, create=None)

A pool which gives out tokens (opaque unique objects), which indicate that the coroutine which holds the token has a right to consume some limited resource.

create()

Generate a new pool item. In order for the pool to function, either this method must be overriden in a subclass or the pool must be constructed with the create argument. It accepts no arguments and returns a single instance of whatever thing the pool is supposed to contain.

In general, create() is called whenever the pool exceeds its previous high-water mark of concurrently-checked-out-items. In other words, in a new pool with min_size of 0, the very first call to get() will result in a call to create(). If the first caller calls put() before some other caller calls get(), then the first item will be returned, and create() will not be called a second time.