hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Staszak <cstas...@gmail.com>
Subject Re: HBase on 1 box? how big?
Date Fri, 15 Jan 2010 23:55:23 GMT
+1 for this feature.

I understand some of the questioning along the lines of "why not use
PostgreSQL/MySQL" for a data store that just runs on one host.

However, the driver for me (and I suspect for a growing number of
people) is to write one piece of code that runs at any scale. For some
uses a single host/jvm makes perfect sense: development, demos or
limited production data size and transaction volume.

Furthermore, this could greatly simplify demos or small scale
deployments on Windows (removing the ssh requirement).

On Fri, Jan 15, 2010 at 2:42 PM, stack <stack@duboce.net> wrote:
> How about we add a 'standalone' argument to bin/hbase?  It'd check the
> hbase-site.xml to see it has right standalone basic config. and then it'd
> pass switches to start all up in the one JVM?
> St.Ack
>
> On Fri, Jan 15, 2010 at 2:27 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> i hadda resolve like 40 files of conflicts :-/
>>
>> what i really need though is a tool so that start-hbase.sh wont do the
>> 'normal' thing and just do hbase-daemon.sh start master when running
>> in standalone mode.
>>
>> -ryan
>>
>> On Fri, Jan 15, 2010 at 2:23 PM, Otis Gospodnetic
>> <otis_gospodnetic@yahoo.com> wrote:
>> > Sounds like a yummy patch, Ryan, if you need another nudge. :)
>> >
>> > Otis
>> >
>> >
>> >
>> > ----- Original Message ----
>> >> From: Ryan Rawson <ryanobjc@gmail.com>
>> >> To: hbase-user@hadoop.apache.org
>> >> Sent: Fri, January 15, 2010 5:00:42 PM
>> >> Subject: Re: HBase on 1 box? how big?
>> >>
>> >> Yes I do plan on releasing a patch, but i need to rebase it to trunk.
>> >> It moves a class from test -> java (ie; the ZK in JVM startup class).
>> >>
>> >> maybe soon?
>> >> -ryan
>> >>
>> >> On Fri, Jan 15, 2010 at 1:57 PM, Andrew Purtell wrote:
>> >> > That would be good for developing disconnected against the API. Any
>> >> > plan on releasing a patch Ryan?
>> >> >
>> >> >   - Andy
>> >> >
>> >> >
>> >> >
>> >> > ----- Original Message ----
>> >> >> From: Ryan Rawson
>> >> >> To: hbase-user@hadoop.apache.org
>> >> >> Sent: Fri, January 15, 2010 1:50:32 PM
>> >> >> Subject: Re: HBase on 1 box? how big?
>> >> >>
>> >> >> You can run HBase on any size of machine all single node, by default
>> >> >> when you start hbase it will store files in /tmp and everything
is in
>> >> >> 1 JVM.  How much data can you jam in there?  I'm not totally
sure,
>> >> >> probably a lot more than you might think, but again limited by
the
>> >> >> disk.  I run it on my mac laptop for example.
>> >> >>
>> >> >> I have a patch that will allow a single JVM including zookeeper,
but
>> >> >> it is locked up in my private git for now. This would get rid of
the
>> >> >> need to ssh localhost just to start local hbase.
>> >> >>
>> >> >> -ryan
>> >> >>
>> >> >> On Fri, Jan 15, 2010 at 1:22 PM, Seth Ladd wrote:
>> >> >> > I agree.  HBase in a box is essentially MySQL.  HBase is
built for
>> a
>> >> cluster.
>> >> >> >
>> >> >> > On Fri, Jan 15, 2010 at 11:17 AM, Andrew Purtell wrote:
>> >> >> >> On that scale, why not use MySQL or Postgres?
>> >> >> >>
>> >> >> >> "HBase in a box" is like "dynamic equilibrium", or "virtual
>> reality", or
>> >> >> >> "jumbo shrimp"... :-)
>> >> >> >>
>> >> >> >>  - Andy
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> ----- Original Message ----
>> >> >> >>> From: Otis Gospodnetic
>> >> >> >>> To: hbase-user@hadoop.apache.org
>> >> >> >>> Sent: Fri, January 15, 2010 12:54:42 PM
>> >> >> >>> Subject: HBase on 1 box? how big?
>> >> >> >>>
>> >> >> >>> Hello,
>> >> >> >>>
>> >> >> >>> I understand running HBase on a single box is kind
of
>> >> >> >>> pointless (thanks Andrew Purtell for the reply about
numbers of
>> >> >> >>> boxes)... but I was wondering what kind of box might
one need to
>> >> >> >>> host/run various HBase/Hadoop processes?
>> >> >> >>>
>> >> >> >>> Imagine I just need to have "HBase in a box", so to
speak. :)
>> >> >> >>>
>> >> >> >>> I understand it depends on the volume on data, DB
structure,
>> request
>> >> >> rates...
>> >> >> >>> I don't have those numbers, but say I want HBase to
have 100M
>> rows with
>> >> >> >>> data from Apache logs and want to run the common web
>> analytics/stats
>> >> >> >>> reports on a nightly basis.
>> >> >> >>>
>> >> >> >>> * Would an EC2 Large Instance suffice?
>> >> >> >>> -- Large Instance 7.5 GB of memory, 4 EC2 Compute
Units (2
>> virtual cores
>> >> >> >>> with 2 EC2 Compute Units each), 850 GB of local instance
storage,
>> 64-bit
>> >> >> >>> platform
>> >> >> >>>
>> >> >> >>> * How about EC2 Small Instance?
>> >> >> >>> -- Small Instance (Default) 1.7 GB of memory, 1 EC2
Compute Unit
>> (1
>> >> virtual
>> >> >> core
>> >> >> >>> with 1 EC2 Compute Unit), 160 GB of local instance
storage,
>> 32-bit
>> >> platform
>> >> >> >>>
>> >> >> >>> Thanks,
>> >> >> >>> Otis
>> >> >> >>> P.S.
>> >> >> >>> hw specs from http://aws.amazon.com/ec2/#instance
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >
>> >
>>
>

Mime
View raw message