tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jihoon Son <ghoon...@gmail.com>
Subject How many task runner instances corresponding to one container?
Date Fri, 25 Apr 2014 07:44:22 GMT
Min, truly apologize for the late response.
I missed your mail..

Here are my answers.
Even though each worker is assigned only one container, it can still
execute multiple tasks simultaneously.
The number of tasks is decided by the configured resource capacity and the
required amount of resources for each task.

You can find the resource configuration for each task in TajoConf as
follows.

...
TASK_DEFAULT_MEMORY("tajo.task.memory-slot-mb.default", 512),
TASK_DEFAULT_DISK("tajo.task.disk-slot.default", 0.5f),
...

Also, the resource configuration of workers can be found in
http://tajo.apache.org/docs/current/configuration/worker_configuration.html.

The number of tasks are determined differently according to the query
execution stage.
If the current stage is leaf (that is, input data are read from disk), disk
amount for each task is primarily considered.
Otherwise, memory amount is primarily considered.

I hope that this explanation will be helpful for you.

Thanks,
Jihoon

2014년 4월 19일 토요일, Min Zhou<coderplay@gmail.com>님이 작성한 메시지:

> Thank you Jihoon,
>
> From the code of TaskRunner.run() and your previous explanation, does that
> means multiple tasks in same node share the same container, but can't be
> executed in parallel?  They will be executed one by one, right? So if I
> have N worker nodes, the max parallelism is N , right?
>
> Thanks,
> Min
>
>
> On Fri, Apr 18, 2014 at 7:07 PM, Jihoon Son <jihoonson@apache.org> wrote:
>
> > Hi Min.
> > That's the exact way that Tajo works.
> > For each exec block, only one container is launched to start a task
> runner
> > in each worker.
> > A task runner can execute multiple tasks of the correspond exec block.
> >
> > Thanks,
> > Jihoon
> > 2014. 4. 19. 오전 9:15에 "Min Zhou" <coderplay@gmail.com>님이 작성:
> >
> > > Hi all,
> > >
> > > It seems that for each container, TajoQueryMaster will launch an task
> > > through rpc. But this task is for multiple QueryUnitAttempts which will
> > be
> > > on the same node with the same ExecutionBlockId, right?  In another
> > words,
> > > in one node, one ExecutionBlockId corresponding to one container,  and
> > may
> > > include multiple tasks, right?
> > >
> > > Thanks,
> > > Min
> > > --
> > > My research interests are distributed systems, parallel computing and
> > > bytecode based virtual machine.
> > >
> > > My profile:
> > > http://www.linkedin.com/in/coderplay
> > > My blog:
> > > http://coderplay.javaeye.com
> > >
> >
>
>
>
> --
> My research interests are distributed systems, parallel computing and
> bytecode based virtual machine.
>
> My profile:
> http://www.linkedin.com/in/coderplay
> My blog:
> http://coderplay.javaeye.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message