hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M. C. Srivas" <mcsri...@gmail.com>
Subject Re: Hadoop Java Versions
Date Fri, 01 Jul 2011 05:06:24 GMT
No worries.

I read Todd's post as asking for elaboration ... sometimes knowing what
another similar system does helps in improving your own.


On Thu, Jun 30, 2011 at 9:47 PM, Ian Holsman <hadoop@holsman.net> wrote:

>
> On Jul 1, 2011, at 2:08 PM, M. C. Srivas wrote:
>
> > On Thu, Jun 30, 2011 at 5:24 PM, Todd Lipcon <todd@cloudera.com> wrote:
> >
> >>
> >> I'd advise you to look at "stock hadoop" again. This used to be true,
> but
> >> was fixed a long while back by HDFS-457 and several followup JIRAs.
> >>
> >> If MapR does something fancier, I'm sure we'd be interested to hear
> about
> >> it
> >> so we can compare the approaches.
> >>
> >> -Todd
> >>
> >>
> > MapR tracks disk responsiveness. In other words, a moving histogram of
> > IO-completion times is maintained internally, and if a disk starts
> getting
> > really slow, it is pre-emptively taken offline so it does not create long
> > tails for running jobs (and the data on the disk is re-replicated using
> > whatever re-replication policy is in place).  One of the benefits of
> > managing the disks directly instead of through ext3 / xfs / or other ...
> >
> > All these stats can be fed into Ganglia (or pushed out centrally via a
> text
> > file that can be pulled out using NFS)  if historical info about disk
> > behavior (and failures) needs to be preserved.
> >
> > - Srivas.
>
> While I am intrigued about how MapR performs internally, I don't think this
> is the forum for it.
> please keep MapR (and other vendor specific discussions) on their
> respective support forums.
>
> Thanks!
>
> Ian.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message