shapepipe.pipeline.worker_handler
shapepipe.pipeline.worker_handler¶
WORKER HANDLER.
This module defines a class for handling pipeline wokers.
- Author
Samuel Farrens <samuel.farrens@cea.fr>
- class WorkerHandler(verbose=True)[source]¶
Bases:
objectWorker Handler.
This class defines the worker to process a given job.
- worker(process, job_name, w_log_name, run_dirs, config, module_config_sec, timeout, module_runner)[source]¶
Worker.
This method defines a worker.
- Parameters
process (numpy.ndarray) – File(s) to be processed
w_log_name (str) – Worker log name
module_runner (function) – Module runner
run_dirs (dict) – Run directories
config (CustomParser) – Configuaration parser instance
module_config_sec (str) – Configuration file section name
timeout (int) – Timeout limit in seconds
- Returns
Worker dictionary
- Return type
- static _set_job_name(num)[source]¶
Set Job Name.
This method creates a job name for a given process number.
- _prepare_worker(process, job_name, timeout, module)[source]¶
Prepare Worker.
This method defines a worker instance dictionary.
- Parameters
Process (str) – File to be processed
config (CustomParser) – Configuaration parser instance
timeout (int) – Timeout limit in seconds
module (str) – Module runner name
- _create_worker_log()[source]¶
Create Worker Log.
This method prepares a logging instance for the worker and logs the worker parameters.
- _run_worker()[source]¶
Run Worker.
This method runs the worker with a given timeout limit and catches the corresponding errors.
- _worker_execution()[source]¶
Worker Execution.
This method executes a worker job and logs the results.
- _run_module()[source]¶
Run Module.
This method runs a module script.
- Raises
RuntimeError – For non-existent module runner