uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Gupta <gupta.v.a...@gmail.com>
Subject Scale out tuning for jobs
Date Wed, 15 Oct 2014 23:11:41 GMT
Hi,
I've been trying to find the options related to configuration of scaleout
of a ducc job.

Thus far the only ones Ive found are:

process_deployments_max:
which limits the maximum number of processes spawned by a ducc job.

At what point does DUCC decide to spawn a new process or spread processing
out to a new node. Is there a tuning parameter for an optimal number of
work items per process spawned? Can the user control this behavior?

For example,
I have a job large enough that DUCC natively spreads it across 2 nodes.
I havent been able to force this job, via a config parameter, to spread
across 4 nodes (or "X" nodes) for faster processing times.

Does anyone know if theres a parameter than can directly control scaleout
in this manner?

Thanks,

-- 
Amit Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message