hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dalia Sobhy <dalia.mohso...@hotmail.com>
Subject RE: Hbase Read/Write throughput measure
Date Sun, 03 Feb 2013 21:05:05 GMT

Hello Mohamed,

I am using ubuntu 12, but the problem is that I was facing some errors while compiling, after
solving them.

After $git clone http://github.com/brianfrankcooper/YCSB.git
Cloning into 'YCSB'...
remote: Counting objects: 2260, done.
remote: Compressing objects: 100% (694/694), done.
Receiving objects:  86% (1952/2260), 764.00 KiB | 51 KiB/s  

It keeps hanging on 86%, so any clue??

Thanksss :D

> From: dontariq@gmail.com
> Date: Mon, 4 Feb 2013 00:17:40 +0530
> Subject: Re: Hbase Read/Write throughput measure
> To: user@hbase.apache.org
> 
> Hello Dalia,
> 
>           Me and Anil are talking about this :
> 
> 1 - git clone http://github.com/brianfrankcooper/YCSB.git
> 2 - cd YCSB/
> 3 - mvn compile
> 
> And not compiling through Java
> 
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
> 
> 
> On Mon, Feb 4, 2013 at 12:14 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com>wrote:
> 
> > Hi anil,
> >
> > How to recompile it ?
> >
> > Sent from my iPhone
> >
> > On 2013-02-03, at 8:00 PM, "anil gupta" <anilgupta84@gmail.com> wrote:
> >
> > > Hi Dalia,
> > >
> > > Recently, i ran into same problem with YCSB. You will need to recompile
> > > YCSB with HBase0.92. Download the maven project from yscb site and
> > > recompile it with HBase0.92.
> > >
> > > HTH,
> > > Anil Gupta
> > >
> > > On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> > >wrote:
> > >
> > >>
> > >> Hello Mohamed,
> > >>
> > >> Yes Hbase is running good and already I have a table and and some
> > records
> > >> there.
> > >>
> > >> I have done what you told me I copied all jar files from lib directory
> > to
> > >> YSCB/db/hbase/lib/ directory. After restart and writing this command
> > >> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
> > >> receive the same error.
> > >>
> > >> But I didn't compile, because already I don't have a .java file
> > >>
> > >> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
> > >> only this "hbase-binding-0.1.4.jar"
> > >>
> > >> So what to do ?
> > >>
> > >>
> > >>> From: dontariq@gmail.com
> > >>> Date: Sun, 3 Feb 2013 22:54:13 +0530
> > >>> Subject: Re: Hbase Read/Write throughput measure
> > >>> To: user@hbase.apache.org
> > >>>
> > >>> Hello Dalia,
> > >>>
> > >>>        Is your Hbase running OK? Copy the hbase jar from your
> > HBASE_HOME
> > >>> and all the jars from your HBASE_HOME/lib/ directory to your
> > >>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
> > >>>
> > >>> Warm Regards,
> > >>> Tariq
> > >>> https://mtariq.jux.com/
> > >>> cloudfront.blogspot.com
> > >>>
> > >>>
> > >>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <
> > dalia.mohsobhy@hotmail.com
> > >>> wrote:
> > >>>
> > >>>>
> > >>>> Hello Mohamed,
> > >>>>
> > >>>> I believe thats not the case.
> > >>>>
> > >>>> check this blog
> > >>
> > http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
> > >>>>
> > >>>> It denotes that this error is due to incompatible Hbase client
and
> > >> server
> > >>>> versions, although I am using Hbase server :0.92 and the server
is the
> > >>>> latest one uploaded by ycsb. So I don't know why this is happening.
> > >>>>
> > >>>> So do you have any idea?
> > >>>>
> > >>>> This is the cmd I am executing:
> > >>>>
> > >>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
> > >> columnfamily="family"
> > >>>> -P workloads/workloada
> > >>>>
> > >>>> java -cp
> > >>
> > /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > >>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> > >>>> columnfamily=family -P workloads/workloada -t
> > >>>> YCSB Client 0.1
> > >>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
> > >> -P
> > >>>> workloads/workloada -t
> > >>>> Loading workload...
> > >>>> Starting test.
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010
05:32
> > >> GMT
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > >> host.name=
> > >>>> master1.medcloud.com
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.version=1.6.0_31
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.vendor=Sun Microsystems Inc.
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>
> > environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>
> > environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.io.tmpdir=/tmp
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:java.compiler=<NA>
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > os.name
> > >>>> =Linux
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:os.arch=amd64
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:os.version=3.2.0-23-generic
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
> > >> user.name
> > >>>> =root
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:user.home=/root
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> > >>>> environment:user.dir=/home/dalia/ycsb-0.1.4
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
> > >> connection,
> > >>>> connectString=192.168.2.3:2181 sessionTimeout=180000
> > >> watcher=hconnection
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
> > >> to
> > >>>> server /192.168.2.3:2181
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
> > >> established
> > >>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
> > >>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> > >>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid
=
> > >>>> 0x13c9b45ac70000a, negotiated timeout = 60000
> > >>>> java.lang.IllegalArgumentException: Not a host:port pair: �
> > >>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
> > >>>>
> > >>>>
> > >>>> my hbase-site.xml:
> > >>>>
> > >>>> <configuration>
> > >>>>  <property>
> > >>>>    <name>hbase.rootdir</name>
> > >>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
> > >>>>
> > >>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
> > >>>>    <description>The directory shared by region servers.
> > >>>>    </description>
> > >>>>  </property>
> > >>>>  <property>
> > >>>>    <name>hbase.master</name>
> > >>>>        <value>192.168.2.3:60000</value>
> > >>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
> > >>>>    <description>The host and port that the HBase master runs
at.
> > >>>>    </description>
> > >>>>  </property>
> > >>>>  <property>
> > >>>>    <name>hbase.zookeeper.quorum</name>
> > >>>>    <value>192.168.2.3</value>
> > >>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
> > >>>>  </property>
> > >>>> </configuration>
> > >>>>
> > >>>> So any help please....
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>>> From: dontariq@gmail.com
> > >>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
> > >>>>> Subject: Re: Hbase Read/Write throughput measure
> > >>>>> To: user@hbase.apache.org
> > >>>>>
> > >>>>> Hello Dalia,
> > >>>>>
> > >>>>>       I am sorry for late response, courtesy hectic schedule.
Add
> > >> the
> > >>>>> hbase-site.xml file to the classpath and see if it works for
you.
> > >>>>>
> > >>>>> Warm Regards,
> > >>>>> Tariq
> > >>>>> https://mtariq.jux.com/
> > >>>>> cloudfront.blogspot.com
> > >>>>>
> > >>>>>
> > >>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
> > >> dalia.mohsobhy@hotmail.com
> > >>>>> wrote:
> > >>>>>
> > >>>>>>
> > >>>>>> I tried yahoo bench mark I got the following error:
> > >>>>>>
> > >>>>>> java.lang.IllegalArgumentException: Not a host:port
> > >>>>>>
> > >>>>>> So any help,
> > >>>>>>
> > >>>>>> Note that I am using hbase 0.92
> > >>>>>>
> > >>>>>> I have downloaded the lastest ycsb from github
> > >>>>>>
> > >>>>>>> From: dalia.mohsobhy@hotmail.com
> > >>>>>>> To: user@hbase.apache.org
> > >>>>>>> Subject: RE: Hbase Read/Write throughput measure
> > >>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> any help ??!!!!!!
> > >>>>>>>
> > >>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
> > >>>>>>>> Subject: Re: Hbase Read/Write throughput measure
> > >>>>>>>> From: jean-marc@spaggiari.org
> > >>>>>>>> To: user@hbase.apache.org
> > >>>>>>>>
> > >>>>>>>> When you will have figure how the command is working,
you will
> > >>>> have to
> > >>>>>>>> understand the output ;)
> > >>>>>>>>
> > >>>>>>>> This is how I just tried it from my HBase directory:
> > >>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
> > >> randomRead
> > >>>> 10
> > >>>>>>>>
> > >>>>>>>> And this is the output (last lines):
> > >>>>>>>> 10066326    56314
> > >>>>>>>> 10171183    56385
> > >>>>>>>> 10276040    56579
> > >>>>>>>> 10380897    56553
> > >>>>>>>>
> > >>>>>>>> Does ask me what it means ;) I have, so far, no
idea.
> > >>>>>>>>
> > >>>>>>>> JM
> > >>>>>>>>
> > >>>>>>>> 2013/2/1, Dalia Sobhy <dalia.mohsobhy@hotmail.com>:
> > >>>>>>>>> Dear Mohamed,
> > >>>>>>>>>
> > >>>>>>>>> I checked this link:
> > >>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > >>>>>>>>>
> > >>>>>>>>> But how to use the commands there is no clue,
note that I am
> > >>>> using
> > >>>>>> cloudera
> > >>>>>>>>> manager.
> > >>>>>>>>>
> > >>>>>>>>> The other yahoo link I couldn't understand
how to use it.
> > >>>>>>>>>
> > >>>>>>>>> So any help please??
> > >>>>>>>>>
> > >>>>>>>>> Sent from my iPad
> > >>>>>>>>>
> > >>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq"
<
> > >> dontariq@gmail.com
> > >>>>>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hello Dalia,
> > >>>>>>>>>>
> > >>>>>>>>>>         I think the easiest way to measure
the read/write
> > >>>>>> throughput is
> > >>>>>>>>>> to use "PerformanceEvaluation" tool that
comes with the
> > >> Hbase
> > >>>>>>>>>> distribution. It spawns a map-reduce job
to do the
> > >> reads/writes
> > >>>> in
> > >>>>>>>>>> parallel. Apart from this there are several
other ways to
> > >>>> benchmark
> > >>>>>> your
> > >>>>>>>>>> Hbase cluster like YSCB <
> > >>>> https://github.com/brianfrankcooper/YCSB>.
> > >>>>>> You
> > >>>>>>>>>> might find this
> > >>>>>>>>>> link<
> > >> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > >>>>>> useful
> > >>>>>>>>>> that talks about Hbase performance.
> > >>>>>>>>>>
> > >>>>>>>>>> HTH
> > >>>>>>>>>>
> > >>>>>>>>>> Warm Regards,
> > >>>>>>>>>> Tariq
> > >>>>>>>>>> https://mtariq.jux.com/
> > >>>>>>>>>> cloudfront.blogspot.com
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > >>>>>>>>>> <dalia.mohsobhy@hotmail.com>wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Dear all,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I want to measure the read/write throughput
for a code on a
> > >>>>>> cluster of
> > >>>>>>>>>>> 10
> > >>>>>>>>>>> nodes. So is there any code or way
to measure it?
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have seen in a cloudera-based presentation
that hbase
> > >>>> read/write
> > >>>>>>>>>>> throughput = millions queries per second.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So any help please??
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Best Regards,
> > >>>>>>>>>>> Dalia Sobhy
> > >
> > >
> > >
> > > --
> > > Thanks & Regards,
> > > Anil Gupta
> >
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message