hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: Sane max storage size for DN
Date Wed, 12 Dec 2012 15:44:33 GMT
Yes it does make sense, depending on how much compute each byte of data
will require on average.  With ordinary Hadoop, it is reasonable to have
half a dozen 2TB drives.  With specialized versions of Hadoop considerably
more can be supported.

>From what you say, it sounds like you are suggesting that your name node
get a part of a single drive with the rest being shared with other virtual
instances or with an OS partition.  That would be a really bad idea for
performance.  Many Hadoop programs are I/O bound so having more than one
spindle is a good thing.

On Wed, Dec 12, 2012 at 7:02 AM, Mohammad Tariq <dontariq@gmail.com> wrote:

> Hello list,
>           I don't know if this question makes any sense, but I would like
> to ask, does it make sense to store 500TB (or more) data in a single DN?If
> yes, then what should be the spec of other parameters *viz*. NN & DN RAM,
> N/W etc?If no, what could be the alternative?
> Many thanks.
> Regards,
>     Mohammad Tariq

View raw message