hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: [Lucene-hadoop Wiki] Update of "FAQ" by DevarajDas
Date Thu, 06 Sep 2007 18:43:52 GMT
Doug Cutting wrote:
> Raghu Angadi wrote:
>> I don't think there is an explanation of why increasing the handlers 
>> proportionally helps (I does help, but it might be a big hammer 
>> approach). I  think ipc Q-length and q management also matters a lot. 
>> I will open a Jira with couple thoughts/explanation/improvements regd 
>> Q mangagement in our IPC.
> Regardless, we should try to make such proportional parameters 
> automatically adjust to cluster size. 

Agreed. I was implying that we should have at least some 
back-of-the-envolope explantions for these formulas. Yes, such 
proportional config would be very useful.

Right now Namenode does not know about the cluster size before starting 
IPC server.


> So instead of having parameters 
> whose values must be proportional to cluster size, we should have 
> parameters that are constants of proportionality (e.g., multiplied by 
> the cluster size to determine the runtime value).  We have a list here 
> of three such parameters and we should thus work to fix each of these to 
> be proportional to cluster size, no?  If today it's handlers and 
> tomorrow it's queues we'll be better served if folks don't have to edit 
> their config files when we make such changes.
> Doug

View raw message