hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: how many severs in a hbase cluster
Date Sat, 22 Jun 2013 17:05:17 GMT
Yeah, I forgot to mention that no. of ZKs should be odd. Perhaps those
parentheses made that statement look like an optional statement. Just to
clarify it was mandatory.

Warm Regards,
Tariq
cloudfront.blogspot.com


On Sat, Jun 22, 2013 at 9:45 PM, Kevin O'dell <kevin.odell@cloudera.com>wrote:

> If you run ZK with a DN/TT/RS please make sure to dedicate a hard drive and
> a core to the ZK process. I have seen many strange occurrences.
> On Jun 22, 2013 12:10 PM, "Jean-Marc Spaggiari" <jean-marc@spaggiari.org>
> wrote:
>
> > You HAVE TO run a ZK3, or else you don't need to have ZK2 and any ZK
> > failure will be an issue. You need to have an odd number of ZK
> > servers...
> >
> > Also, if you don't run MR jobs, you don't need the TT and JT... Else,
> > everything below is correct. But there is many other options, all
> > depend on your needs and the hardware you have ;)
> >
> > JM
> >
> > 2013/6/22 Mohammad Tariq <dontariq@gmail.com>:
> > > With 8 machines you can do something like this :
> > >
> > > Machine 1 - NN+JT
> > > Machine 2 - SNN+ZK1
> > > Machine 3 - HM+ZK2
> > > Machine 4-8 - DN+TT+RS
> > > (You can run ZK3 on a slave node with some additional memory).
> > >
> > > DN and RS run on the same machine. Although RSs are said to hold the
> > data,
> > > the data is actually stored in DNs. Replication is managed at HDFS
> level.
> > > You don't have to worry about that.
> > >
> > > You can visit this link <
> http://hbase.apache.org/book/perf.writing.html>
> > to
> > > see how to write efficiently into HBase. With a small field there
> should
> > > not be any problem except storage and increased metadata, as you'll
> have
> > > many small cells. If possible club several small fields into one and
> put
> > > them together in one cell.
> > >
> > > HTH
> > >
> > > Warm Regards,
> > > Tariq
> > > cloudfront.blogspot.com
> > >
> > >
> > > On Sat, Jun 22, 2013 at 8:31 PM, myhbase <myhbase@126.com> wrote:
> > >
> > >> Thanks for your response.
> > >>
> > >> Now if 5 servers are enough, how can I install  and configure my
> nodes?
> > If
> > >> I need 3 replicas in case data loss, I should at least have 3
> > datanodes, we
> > >> still have namenode, regionserver and HMaster nodes, zookeeper nodes,
> > some
> > >> of them must be installed in the same machine. The datanode seems the
> > disk
> > >> IO sensitive node while region server is the mem sensitive, can I
> > install
> > >> them in the same machine? Any suggestion on the deployment plan?
> > >>
> > >> My business requirement is that the write is much more than read(7:3),
> > and
> > >> I have another concern that I have a field which will have the 8~15KB
> in
> > >>  data size, I am not sure, there will be any problem in hbase when it
> > runs
> > >> compaction and split in regions.
> > >>
> > >>  Oh, you already have heavyweight's input :).
> > >>>
> > >>> Thanks JM.
> > >>>
> > >>> Warm Regards,
> > >>> Tariq
> > >>> cloudfront.blogspot.com
> > >>>
> > >>>
> > >>> On Sat, Jun 22, 2013 at 8:05 PM, Mohammad Tariq <dontariq@gmail.com>
> > >>> wrote:
> > >>>
> > >>>  Hello there,
> > >>>>
> > >>>>          IMHO, 5-8 servers are sufficient enough to start with.
But
> > it's
> > >>>> all relative to the data you have and the intensity of your
> > reads/writes.
> > >>>> You should have different strategies though, based on whether it's
> > 'read'
> > >>>> or 'write'. You actually can't define 'big' in absolute terms.
My
> > cluster
> > >>>> might be big for me, but for someone else it might still be not
big
> > >>>> enough
> > >>>> or for someone it might be very big. Long story short it depends
on
> > your
> > >>>> needs. If you are able to achieve your goal with 5-8 RSs, then
> having
> > >>>> more
> > >>>> machines will be a wastage, I think.
> > >>>>
> > >>>> But you should always keep in mind that HBase is kinda greedy when
> it
> > >>>> comes to memory. For a decent load 4G is sufficient, IMHO. But
it
> > again
> > >>>> depends on operations you are gonna perform. If you have large
> > clusters
> > >>>> where you are planning to run MR jobs frequently you are better
off
> > with
> > >>>> additional 2G.
> > >>>>
> > >>>>
> > >>>> Warm Regards,
> > >>>> Tariq
> > >>>> cloudfront.blogspot.com
> > >>>>
> > >>>>
> > >>>> On Sat, Jun 22, 2013 at 7:51 PM, myhbase <myhbase@126.com>
wrote:
> > >>>>
> > >>>>  Hello All,
> > >>>>>
> > >>>>> I learn hbase almost from papers and books, according to my
> > >>>>> understanding, HBase is the kind of architecture which is more
> > appliable
> > >>>>> to a big cluster. We should have many HDFS nodes, and many
> > HBase(region
> > >>>>> server) nodes. If we only have several severs(5-8), it seems
hbase
> is
> > >>>>> not a good choice, please correct me if I am wrong. In addition,
> how
> > >>>>> many nodes usually we can start to consider the hbase solution
and
> > how
> > >>>>> about the physic mem size and other hardware resource in each
node,
> > any
> > >>>>> reference document or cases? Thanks.
> > >>>>>
> > >>>>> --Ning
> > >>>>>
> > >>>>>
> > >>>>>
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message