hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Occasional regionserver crashes following socket errors writing to HDFS
Date Fri, 11 May 2012 03:28:08 GMT
On Thu, May 10, 2012 at 6:28 PM, Michael Segel
<michael_segel@hotmail.com> wrote:
> That section was written by Doug after he and I had the same debate man moons ago.

I'm not sure that is correct.  If you git blame that section, you'll
see that stack and andrew are the authors and that the edits were made
in 2009 and 2010.

There is this section in the book but it doesn't seem to have the
benefit of your input:

> While I can't say with absolute certainty that you shouldn't use a reducer, I can say
is that every situation where I have seen a M/R where you are writing to HBase, you end up
not wanting to use a reducer. If you want a clear and concise statement you can say that the
rule of thumb is that you don't want to use a reducer and that cases where you would need
to first use a reducer are the rare exception.

Please file an issue w/ a patch.  It'd be good to get your experience
into the doc.

> The reason I ask people to think about this topic is that unless you have a really good
foundation in databases, not relying on a reducer is a bit counter intuitive. (Which is why
I said that you really need to clear your mind and focus on this issue. )

Lets make it so that if you don't have a foundation in dbs, if you
read the doc., you won't need such a background to get the best of

> PS. If you care to read the thread, I didn't become condescending until a certain individual
piped up about how refactoring the M/R was a 'distraction' to the issue at hand.
> Not to mention his flip response w the Google paper?

There are a few problems w/ the above.

+ You presume I did not read the thread before responding
+ That the condescending tone started after Dave's intercessions (I
was not referring to this thread only).

Michael, fellas like you help move the hbase story along.   Generally,
I see that you do a great job in this forum and in others.  In my
previous note, I was just trying to give a pointer that what you might
consider jest, others can read as condescending or sarcasm.


View raw message