tajo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schwabe <Christian.Schw...@gmx.com>
Subject Re: Worker Resources
Date Wed, 17 Sep 2014 12:40:42 GMT
Hello guys,

The log has been strangely cut off, I noticed straight. Attached again the correct and complete
log.

Best regards,

Chris

Am 17.09.2014 um 13:45 schrieb Christian Schwabe <Christian.Schwabe@gmx.com>:

> Hello guys,
> 
> I promised I do not pretend to solve this problem together with you. I would like to
redeem and more questions that bring us closer to a solution, hopefully this promise. My Macbook
Pro has 8 GB of RAM and 4 cores available. My configuration in tajo-site.xml looks like this:
> 
> <configuration>
> <property>
>   <name>tajo.resource-tracker.heartbeat.timeout-secs</name>
>   <value>240000</value>  // or, your own longer value. default is 120*1000
(2 minutes)
> </property>
> <property>
>   <name>tajo.worker.resource.tajo.worker.resource.cpu-cores</name>
>   <value>4</value>
> </property>
>  <property>
>   <name>tajo.worker.resource.memory-mb</name>
>   <value>2048</value>
> </property>
> </configuration>
> 
> 
> Actually, I would now expect a fourfold parallel processing. However, I can only see
a maximum of three tasks at once with the status"running". With this configuration, I can
still establish the following: Tajo processed, judging by the progress indicator is now more
data than before. Up to now, only about 75% possible I land now at about 90% until Tajo crashed
again.
> I attached a log for this run. Also the following session variables were applied for
this run:
> tajo> \set JOIN_TASK_INPUT_SIZE 64
> tajo> \set JOIN_PER_SHUFFLE_SIZE 64
> 
> Do you have any ideas I can try?
> 
> 
> 
> Best regards and very thankful,
> Chris
> 
> 
> 
> <tajo-chris-master-christians-mbp.fritz.box.log><tajo-chris-worker-christians-mbp.fritz.box.log>


Mime
View raw message