jobqueues.celeryqueue module#

class jobqueues.celeryqueue.CeleryQueue(_configapp=None, _configfile=None, _logger=True, _checkWorkers=True)#

Bases: LocalGPUQueue

Beta: Queue with support for Celery local queueing

Start a celery server with: >>> docker run -d -p 5462:5672 rabbitmq >>> celery –app=jobqueues.celeryfiles.celery worker –loglevel=info -Q gpu -c 4 -n gpu@%h >>> celery –app=jobqueues.celeryfiles.celery worker –loglevel=info -Q cpu -c 10 -n cpu@%h

inprogress()#

Get the number of simulations in progress

Returns the sum of the number of running and queued workunits of the specific group in the engine.

Example

>>> app.inprogress()

property memory#: Subclasses need to have this property. This property is expected to return a integer in MiB

property ncpu#: Subclasses need to have this property

property ngpu#: Subclasses need to have this property

retrieve()#

Retrieves a list of jobs that have completed since the last call

Example

>>> comp = app.retrieve()

stop()#: Subclasses need to implement this method

submit(dirs, commands=None)#

Queue for execution all of the jobs in the passed list of directories

Queues all work units in a given directory list with the options given in the constructor opt.

Parameters:: mydirs (list of str) – A list or ndarray of directory paths

Examples

>>> app.submit(glob('input/e2*/'))