hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject Re: Hbase Read/Write throughput measure
Date Sun, 03 Feb 2013 17:19:18 GMT
Hi Dalia,

Recently, i ran into the same problem with yscb. You will have to recompile
YCSB by yourself by downloading the maven project. The problem is that YCSB
is not compiled with 0.92 even though they claim to do so. So, once you
locally compile YCSB with HBase0.92 then you should be good to go.

HTH,
Anil Gupta

On Sun, Feb 3, 2013 at 8:48 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com>wrote:

>
> Hello Mohamed,
>
> I believe thats not the case.
>
> check this blog
> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>
> It denotes that this error is due to incompatible Hbase client and server
> versions, although I am using Hbase server :0.92 and the server is the
> latest one uploaded by ycsb. So I don't know why this is happening.
>
> So do you have any idea?
>
> This is the cmd I am executing:
>
> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p columnfamily="family"
> -P workloads/workloada
>
> java -cp
> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
> columnfamily=family -P workloads/workloada -t
> YCSB Client 0.1
> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family -P
> workloads/workloada -t
> Loading workload...
> Starting test.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:host.name=
> master1.medcloud.com
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_31
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
> =Linux
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:os.version=3.2.0-23-generic
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:user.name
> =root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.home=/root
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
> environment:user.dir=/home/dalia/ycsb-0.1.4
> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=192.168.2.3:2181 sessionTimeout=180000 watcher=hconnection
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> server /192.168.2.3:2181
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection established
> to master1.medcloud.com/192.168.2.3:2181, initiating session
> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
> 0x13c9b45ac70000a, negotiated timeout = 60000
> java.lang.IllegalArgumentException: Not a host:port pair: �
> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>
>
> my hbase-site.xml:
>
> <configuration>
>   <property>
>     <name>hbase.rootdir</name>
>    <value>hdfs://192.168.2.3:20001/hbase</value>
>
> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>     <description>The directory shared by region servers.
>     </description>
>   </property>
>   <property>
>     <name>hbase.master</name>
>         <value>192.168.2.3:60000</value>
>     <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>     <description>The host and port that the HBase master runs at.
>     </description>
>   </property>
>   <property>
>     <name>hbase.zookeeper.quorum</name>
>     <value>192.168.2.3</value>
>     <!--<value><HBASE_MASTER_SERVER></value>-->
>   </property>
> </configuration>
>
> So any help please....
>
> Thanks,
>
> > From: dontariq@gmail.com
> > Date: Sun, 3 Feb 2013 13:54:28 +0530
> > Subject: Re: Hbase Read/Write throughput measure
> > To: user@hbase.apache.org
> >
> > Hello Dalia,
> >
> >        I am sorry for late response, courtesy hectic schedule. Add the
> > hbase-site.xml file to the classpath and see if it works for you.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
> >wrote:
> >
> > >
> > > I tried yahoo bench mark I got the following error:
> > >
> > > java.lang.IllegalArgumentException: Not a host:port
> > >
> > > So any help,
> > >
> > > Note that I am using hbase 0.92
> > >
> > > I have downloaded the lastest ycsb from github
> > >
> > > > From: dalia.mohsobhy@hotmail.com
> > > > To: user@hbase.apache.org
> > > > Subject: RE: Hbase Read/Write throughput measure
> > > > Date: Sat, 2 Feb 2013 16:47:43 +0200
> > > >
> > > >
> > > > any help ??!!!!!!
> > > >
> > > > > Date: Fri, 1 Feb 2013 17:15:45 -0500
> > > > > Subject: Re: Hbase Read/Write throughput measure
> > > > > From: jean-marc@spaggiari.org
> > > > > To: user@hbase.apache.org
> > > > >
> > > > > When you will have figure how the command is working, you will
> have to
> > > > > understand the output ;)
> > > > >
> > > > > This is how I just tried it from my HBase directory:
> > > > > bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation randomRead
> 10
> > > > >
> > > > > And this is the output (last lines):
> > > > > 10066326    56314
> > > > > 10171183    56385
> > > > > 10276040    56579
> > > > > 10380897    56553
> > > > >
> > > > > Does ask me what it means ;) I have, so far, no idea.
> > > > >
> > > > > JM
> > > > >
> > > > > 2013/2/1, Dalia Sobhy <dalia.mohsobhy@hotmail.com>:
> > > > > > Dear Mohamed,
> > > > > >
> > > > > > I checked this link:
> > > > > > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
> > > > > >
> > > > > > But how to use the commands there is no clue, note that I am
> using
> > > cloudera
> > > > > > manager.
> > > > > >
> > > > > > The other yahoo link I couldn't understand how to use it.
> > > > > >
> > > > > > So any help please??
> > > > > >
> > > > > > Sent from my iPad
> > > > > >
> > > > > > On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <dontariq@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > >> Hello Dalia,
> > > > > >>
> > > > > >>          I think the easiest way to measure the read/write
> > > throughput is
> > > > > >> to use "PerformanceEvaluation" tool that comes with the
Hbase
> > > > > >> distribution. It spawns a map-reduce job to do the reads/writes
> in
> > > > > >> parallel. Apart from this there are several other ways to
> benchmark
> > > your
> > > > > >> Hbase cluster like YSCB <
> https://github.com/brianfrankcooper/YCSB>.
> > > You
> > > > > >> might find this
> > > > > >> link<http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
> > > useful
> > > > > >> that talks about Hbase performance.
> > > > > >>
> > > > > >> HTH
> > > > > >>
> > > > > >> Warm Regards,
> > > > > >> Tariq
> > > > > >> https://mtariq.jux.com/
> > > > > >> cloudfront.blogspot.com
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
> > > > > >> <dalia.mohsobhy@hotmail.com>wrote:
> > > > > >>
> > > > > >>>
> > > > > >>> Dear all,
> > > > > >>>
> > > > > >>> I want to measure the read/write throughput for a code
on a
> > > cluster of
> > > > > >>> 10
> > > > > >>> nodes. So is there any code or way to measure it?
> > > > > >>>
> > > > > >>> I have seen in a cloudera-based presentation that hbase
> read/write
> > > > > >>> throughput = millions queries per second.
> > > > > >>>
> > > > > >>> So any help please??
> > > > > >>>
> > > > > >>> Thanks
> > > > > >>>
> > > > > >>>
> > > > > >>> Best Regards,
> > > > > >>> Dalia Sobhy
> > > > > >>>
> > > > > >
> > > >
> > >
> > >
>
>



-- 
Thanks & Regards,
Anil Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message