hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Hbase Read/Write throughput measure
Date Sun, 03 Feb 2013 21:18:25 GMT
it may take some time depending on your connection. wait for sometime. if
you still find some problem you can visit this
link<http://cloudfront.blogspot.in/2013/02/how-to-benchmark-hbase-using-ycsb.html#.UQ7Tu5vS8s0>

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, Feb 4, 2013 at 2:35 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com>wrote:

>
> Hello Mohamed,
>
> I am using ubuntu 12, but the problem is that I was facing some errors
> while compiling, after solving them.
>
> After $git clone http://github.com/brianfrankcooper/YCSB.git
> Cloning into 'YCSB'...
> remote: Counting objects: 2260, done.
> remote: Compressing objects: 100% (694/694), done.
> Receiving objects:  86% (1952/2260), 764.00 KiB | 51 KiB/s
>
> It keeps hanging on 86%, so any clue??
>
> Thanksss :D
>
> > From: dontariq@gmail.com
> > Date: Mon, 4 Feb 2013 00:17:40 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >           Me and Anil are talking about this :
> >
> > 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> > 2 - cd YCSB/
> > 3 - mvn compile
> >
> > And not compiling through Java
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > > Hi anil,
> > >
> > > How to recompile it ?
> > >
> > > Sent from my iPhone
> > >
> > > On 2013-02-03, at 8:00 PM, "anil gupta" <anilgupta84@gmail.com> wrote:
> > >
> > > > Hi Dalia,
> > > >
> > > > Recently, i ran into same problem with YCSB. You will need to
> recompile
> > > > YCSB with HBase0.92. Download the maven project from yscb site and
> > > > recompile it with HBase0.92.
> > > >
> > > > HTH,
> > > > Anil Gupta
> > > >
> > > > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <
> dalia.mohsobhy@hotmail.com
> > > >wrote:
> > > >
> > > >>
> > > >> Hello Mohamed,
> > > >>
> > > >> Yes Hbase is running good and already I have a table and and some
> > > records
> > > >> there.
> > > >>
> > > >> I have done what you told me I copied all jar files from lib
> directory
> > > to
> > > >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> > > >> "./bin/ycsb run hbase -p columnfamily=family -P
> workloads/workloada", I
> > > >> receive the same error.
> > > >>
> > > >> But I didn't compile, because already I don't have a .java file
> > > >>
> > > >> ycsb-0.1.4 directory doesn't contain hbase client java file, it
> contains
> > > >> only this "hbase-binding-0.1.4.jar"
> > > >>
> > > >> So what to do ?
> > > >>
> > > >>
> > > >>> From: dontariq@gmail.com
> > > >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> > > >>> Subject: Re: Hbase Read/Write throughput measure
> > > >>> To: user@hbase.apache.org
> > > >>>
> > > >>> Hello Dalia,
> > > >>>
> > > >>>        Is your Hbase running OK? Copy the hbase jar from your
> > > HBASE_HOME
> > > >>> and all the jars from your HBASE_HOME/lib/ directory to your
> > > >>> YSCB/db/hbase/lib/ directory, compile it again and restart
> everything.
> > > >>>
> > > >>> Warm Regards,
> > > >>> Tariq
> > > >>> https://mtariq.jux.com/
> > > >>> cloudfront.blogspot.com
> > > >>>
> > > >>>
> > > >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> > > dalia.mohsobhy@hotmail.com
> > > >>> wrote:
> > > >>>
> > > >>>>
> > > >>>> Hello Mohamed,
> > > >>>>
> > > >>>> I believe thats not the case.
> > > >>>>
> > > >>>> check this blog
> > > >>
> > >
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > > >>>>
> > > >>>> It denotes that this error is due to incompatible Hbase client
and
> > > >> server
> > > >>>> versions, although I am using Hbase server :0.92 and the server
> is the
> > > >>>> latest one uploaded by ycsb. So I don't know why this is
> happening.
> > > >>>>
> > > >>>> So do you have any idea?
> > > >>>>
> > > >>>> This is the cmd I am executing:
> > > >>>>
> > > >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> > > >> columnfamily="family"
> > > >>>> -P workloads/workloada
> > > >>>>
> > > >>>> java -cp
> > > >>
> > >
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > > >>>> columnfamily=family -P workloads/workloada -t
> > > >>>> YCSB Client 0.1
> > > >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p
> columnfamily=family
> > > >> -P
> > > >>>> workloads/workloada -t
> > > >>>> Loading workload...
> > > >>>> Starting test.
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010
> 05:32
> > > >> GMT
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > >> host.name=
> > > >>>> master1.medcloud.com
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.version=1.6.0_31
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.vendor=Sun Microsystems Inc.
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>
> > >
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>
> > >
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.io.tmpdir=/tmp
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:java.compiler=<NA>
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > os.name
> > > >>>> =Linux
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:os.arch=amd64
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:os.version=3.2.0-23-generic
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > > >> user.name
> > > >>>> =root
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:user.home=/root
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > > >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> > > >> connection,
> > > >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> > > >> watcher=hconnection
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket
> connection
> > > >> to
> > > >>>> server /192.168.2.3:2181
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> > > >> established
> > > >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> > > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > > >>>> complete on server master1.medcloud.com/192.168.2.3:2181,
> sessionid =
> > > >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> > > >>>> java.lang.IllegalArgumentException: Not a host:port pair:
�
> > > >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > > >>>>
> > > >>>>
> > > >>>> my hbase-site.xml:
> > > >>>>
> > > >>>> <configuration>
> > > >>>>  <property>
> > > >>>>    <name>hbase.rootdir</name>
> > > >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> > > >>>>
> > > >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > > >>>>    <description>The directory shared by region servers.
> > > >>>>    </description>
> > > >>>>  </property>
> > > >>>>  <property>
> > > >>>>    <name>hbase.master</name>
> > > >>>>        <value>192.168.2.3:60000</value>
> > > >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > > >>>>    <description>The host and port that the HBase master
runs at.
> > > >>>>    </description>
> > > >>>>  </property>
> > > >>>>  <property>
> > > >>>>    <name>hbase.zookeeper.quorum</name>
> > > >>>>    <value>192.168.2.3</value>
> > > >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> > > >>>>  </property>
> > > >>>> </configuration>
> > > >>>>
> > > >>>> So any help please....
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>>> From: dontariq@gmail.com
> > > >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> > > >>>>> Subject: Re: Hbase Read/Write throughput measure
> > > >>>>> To: user@hbase.apache.org
> > > >>>>>
> > > >>>>> Hello Dalia,
> > > >>>>>
> > > >>>>>       I am sorry for late response, courtesy hectic schedule.
Add
> > > >> the
> > > >>>>> hbase-site.xml file to the classpath and see if it works
for you.
> > > >>>>>
> > > >>>>> Warm Regards,
> > > >>>>> Tariq
> > > >>>>> https://mtariq.jux.com/
> > > >>>>> cloudfront.blogspot.com
> > > >>>>>
> > > >>>>>
> > > >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> > > >> dalia.mohsobhy@hotmail.com
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>>
> > > >>>>>> I tried yahoo bench mark I got the following error:
> > > >>>>>>
> > > >>>>>> java.lang.IllegalArgumentException: Not a host:port
> > > >>>>>>
> > > >>>>>> So any help,
> > > >>>>>>
> > > >>>>>> Note that I am using hbase 0.92
> > > >>>>>>
> > > >>>>>> I have downloaded the lastest ycsb from github
> > > >>>>>>
> > > >>>>>>> From: dalia.mohsobhy@hotmail.com
> > > >>>>>>> To: user@hbase.apache.org
> > > >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> > > >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> any help ??!!!!!!
> > > >>>>>>>
> > > >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> > > >>>>>>>> From: jean-marc@spaggiari.org
> > > >>>>>>>> To: user@hbase.apache.org
> > > >>>>>>>>
> > > >>>>>>>> When you will have figure how the command
is working, you will
> > > >>>> have to
> > > >>>>>>>> understand the output ;)
> > > >>>>>>>>
> > > >>>>>>>> This is how I just tried it from my HBase
directory:
> > > >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> > > >> randomRead
> > > >>>> 10
> > > >>>>>>>>
> > > >>>>>>>> And this is the output (last lines):
> > > >>>>>>>> 10066326    56314
> > > >>>>>>>> 10171183    56385
> > > >>>>>>>> 10276040    56579
> > > >>>>>>>> 10380897    56553
> > > >>>>>>>>
> > > >>>>>>>> Does ask me what it means ;) I have, so far,
no idea.
> > > >>>>>>>>
> > > >>>>>>>> JM
> > > >>>>>>>>
> > > >>>>>>>> 2013/2/1, Dalia Sobhy <dalia.mohsobhy@hotmail.com>:
> > > >>>>>>>>> Dear Mohamed,
> > > >>>>>>>>>
> > > >>>>>>>>> I checked this link:
> > > >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > >>>>>>>>>
> > > >>>>>>>>> But how to use the commands there is no
clue, note that I am
> > > >>>> using
> > > >>>>>> cloudera
> > > >>>>>>>>> manager.
> > > >>>>>>>>>
> > > >>>>>>>>> The other yahoo link I couldn't understand
how to use it.
> > > >>>>>>>>>
> > > >>>>>>>>> So any help please??
> > > >>>>>>>>>
> > > >>>>>>>>> Sent from my iPad
> > > >>>>>>>>>
> > > >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad
Tariq" <
> > > >> dontariq@gmail.com
> > > >>>>>
> > > >>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Hello Dalia,
> > > >>>>>>>>>>
> > > >>>>>>>>>>         I think the easiest way to
measure the read/write
> > > >>>>>> throughput is
> > > >>>>>>>>>> to use "PerformanceEvaluation" tool
that comes with the
> > > >> Hbase
> > > >>>>>>>>>> distribution. It spawns a map-reduce
job to do the
> > > >> reads/writes
> > > >>>> in
> > > >>>>>>>>>> parallel. Apart from this there are
several other ways to
> > > >>>> benchmark
> > > >>>>>> your
> > > >>>>>>>>>> Hbase cluster like YSCB <
> > > >>>> https://github.com/brianfrankcooper/YCSB>.
> > > >>>>>> You
> > > >>>>>>>>>> might find this
> > > >>>>>>>>>> link<
> > > >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > >>>>>> useful
> > > >>>>>>>>>> that talks about Hbase performance.
> > > >>>>>>>>>>
> > > >>>>>>>>>> HTH
> > > >>>>>>>>>>
> > > >>>>>>>>>> Warm Regards,
> > > >>>>>>>>>> Tariq
> > > >>>>>>>>>> https://mtariq.jux.com/
> > > >>>>>>>>>> cloudfront.blogspot.com
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia
Sobhy
> > > >>>>>>>>>> <dalia.mohsobhy@hotmail.com>wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Dear all,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I want to measure the read/write
throughput for a code on a
> > > >>>>>> cluster of
> > > >>>>>>>>>>> 10
> > > >>>>>>>>>>> nodes. So is there any code or
way to measure it?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I have seen in a cloudera-based
presentation that hbase
> > > >>>> read/write
> > > >>>>>>>>>>> throughput = millions queries
per second.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> So any help please??
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>> Dalia Sobhy
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks & Regards,
> > > > Anil Gupta
> > >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message