hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dalia Sobhy <dalia.mohso...@hotmail.com>
Subject Re: Hbase Read/Write throughput measure
Date Sun, 03 Feb 2013 18:44:24 GMT
Hi anil,

How to recompile it ?

Sent from my iPhone

On 2013-02-03, at 8:00 PM, "anil gupta" <anilgupta84@gmail.com> wrote:

> Hi Dalia,
> 
> Recently, i ran into same problem with YCSB. You will need to recompile
> YCSB with HBase0.92. Download the maven project from yscb site and
> recompile it with HBase0.92.
> 
> HTH,
> Anil Gupta
> 
> On Sun, Feb 3, 2013 at 9:49 AM, Dalia Sobhy <dalia.mohsobhy@hotmail.com>wrote:
> 
>> 
>> Hello Mohamed,
>> 
>> Yes Hbase is running good and already I have a table and and some records
>> there.
>> 
>> I have done what you told me I copied all jar files from lib directory to
>> YSCB/db/hbase/lib/ directory. After restart and writing this command
>> "./bin/ycsb run hbase -p columnfamily=family -P workloads/workloada", I
>> receive the same error.
>> 
>> But I didn't compile, because already I don't have a .java file
>> 
>> ycsb-0.1.4 directory doesn't contain hbase client java file, it contains
>> only this "hbase-binding-0.1.4.jar"
>> 
>> So what to do ?
>> 
>> 
>>> From: dontariq@gmail.com
>>> Date: Sun, 3 Feb 2013 22:54:13 +0530
>>> Subject: Re: Hbase Read/Write throughput measure
>>> To: user@hbase.apache.org
>>> 
>>> Hello Dalia,
>>> 
>>>        Is your Hbase running OK? Copy the hbase jar from your HBASE_HOME
>>> and all the jars from your HBASE_HOME/lib/ directory to your
>>> YSCB/db/hbase/lib/ directory, compile it again and restart everything.
>>> 
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>> 
>>> 
>>> On Sun, Feb 3, 2013 at 10:18 PM, Dalia Sobhy <dalia.mohsobhy@hotmail.com
>>> wrote:
>>> 
>>>> 
>>>> Hello Mohamed,
>>>> 
>>>> I believe thats not the case.
>>>> 
>>>> check this blog
>> http://stackoverflow.com/questions/11649824/hbase-error-not-a-hostport-pair
>>>> 
>>>> It denotes that this error is due to incompatible Hbase client and
>> server
>>>> versions, although I am using Hbase server :0.92 and the server is the
>>>> latest one uploaded by ycsb. So I don't know why this is happening.
>>>> 
>>>> So do you have any idea?
>>>> 
>>>> This is the cmd I am executing:
>>>> 
>>>> root@master1:~/ycsb-0.1.4# ./bin/ycsb run hbase -p
>> columnfamily="family"
>>>> -P workloads/workloada
>>>> 
>>>> java -cp
>> /home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>>>> com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.HBaseClient -p
>>>> columnfamily=family -P workloads/workloada -t
>>>> YCSB Client 0.1
>>>> Command line: -db com.yahoo.ycsb.db.HBaseClient -p columnfamily=family
>> -P
>>>> workloads/workloada -t
>>>> Loading workload...
>>>> Starting test.
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
>> GMT
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> host.name=
>>>> master1.medcloud.com
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.version=1.6.0_31
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.vendor=Sun Microsystems Inc.
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.home=/usr/lib/jvm/j2sdk1.6-oracle/jre
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> environment:java.class.path=/home/dalia/ycsb-0.1.4/gemfire-binding/conf:/home/dalia/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/home/dalia/ycsb-0.1.4/hbase-binding/conf:/home/dalia/ycsb-0.1.4/jdbc-binding/conf:/home/dalia/ycsb-0.1.4/voldemort-binding/conf:/home/dalia/ycsb-0.1.4/infinispan-binding/conf:/home/dalia/ycsb-0.1.4/nosqldb-binding/conf:/home/dalia/ycsb-0.1.4/core/lib/core-0.1.4.jar
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>> environment:java.library.path=/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64/server:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/amd64:/usr/lib/jvm/j2sdk1.6-oracle/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.io.tmpdir=/tmp
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:java.compiler=<NA>
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:os.name
>>>> =Linux
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:os.arch=amd64
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:os.version=3.2.0-23-generic
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client environment:
>> user.name
>>>> =root
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:user.home=/root
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Client
>>>> environment:user.dir=/home/dalia/ycsb-0.1.4
>>>> 13/02/03 18:42:24 INFO zookeeper.ZooKeeper: Initiating client
>> connection,
>>>> connectString=192.168.2.3:2181 sessionTimeout=180000
>> watcher=hconnection
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Opening socket connection
>> to
>>>> server /192.168.2.3:2181
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Socket connection
>> established
>>>> to master1.medcloud.com/192.168.2.3:2181, initiating session
>>>> 13/02/03 18:42:24 INFO zookeeper.ClientCnxn: Session establishment
>>>> complete on server master1.medcloud.com/192.168.2.3:2181, sessionid =
>>>> 0x13c9b45ac70000a, negotiated timeout = 60000
>>>> java.lang.IllegalArgumentException: Not a host:port pair: �
>>>> 1799@master1.medcloud.commaster1.medcloud.com,60020,1359814720801
>>>> 
>>>> 
>>>> my hbase-site.xml:
>>>> 
>>>> <configuration>
>>>>  <property>
>>>>    <name>hbase.rootdir</name>
>>>>   <value>hdfs://192.168.2.3:20001/hbase</value>
>>>> 
>>>> <!--   <value>hdfs://<HBASE_MASTER_SERVER>:20001/hbase</value>-->
>>>>    <description>The directory shared by region servers.
>>>>    </description>
>>>>  </property>
>>>>  <property>
>>>>    <name>hbase.master</name>
>>>>        <value>192.168.2.3:60000</value>
>>>>    <!--<value><HBASE_MASTER_SERVER>:60000</value>-->
>>>>    <description>The host and port that the HBase master runs at.
>>>>    </description>
>>>>  </property>
>>>>  <property>
>>>>    <name>hbase.zookeeper.quorum</name>
>>>>    <value>192.168.2.3</value>
>>>>    <!--<value><HBASE_MASTER_SERVER></value>-->
>>>>  </property>
>>>> </configuration>
>>>> 
>>>> So any help please....
>>>> 
>>>> Thanks,
>>>> 
>>>>> From: dontariq@gmail.com
>>>>> Date: Sun, 3 Feb 2013 13:54:28 +0530
>>>>> Subject: Re: Hbase Read/Write throughput measure
>>>>> To: user@hbase.apache.org
>>>>> 
>>>>> Hello Dalia,
>>>>> 
>>>>>       I am sorry for late response, courtesy hectic schedule. Add
>> the
>>>>> hbase-site.xml file to the classpath and see if it works for you.
>>>>> 
>>>>> Warm Regards,
>>>>> Tariq
>>>>> https://mtariq.jux.com/
>>>>> cloudfront.blogspot.com
>>>>> 
>>>>> 
>>>>> On Sun, Feb 3, 2013 at 3:50 AM, Dalia Sobhy <
>> dalia.mohsobhy@hotmail.com
>>>>> wrote:
>>>>> 
>>>>>> 
>>>>>> I tried yahoo bench mark I got the following error:
>>>>>> 
>>>>>> java.lang.IllegalArgumentException: Not a host:port
>>>>>> 
>>>>>> So any help,
>>>>>> 
>>>>>> Note that I am using hbase 0.92
>>>>>> 
>>>>>> I have downloaded the lastest ycsb from github
>>>>>> 
>>>>>>> From: dalia.mohsobhy@hotmail.com
>>>>>>> To: user@hbase.apache.org
>>>>>>> Subject: RE: Hbase Read/Write throughput measure
>>>>>>> Date: Sat, 2 Feb 2013 16:47:43 +0200
>>>>>>> 
>>>>>>> 
>>>>>>> any help ??!!!!!!
>>>>>>> 
>>>>>>>> Date: Fri, 1 Feb 2013 17:15:45 -0500
>>>>>>>> Subject: Re: Hbase Read/Write throughput measure
>>>>>>>> From: jean-marc@spaggiari.org
>>>>>>>> To: user@hbase.apache.org
>>>>>>>> 
>>>>>>>> When you will have figure how the command is working, you
will
>>>> have to
>>>>>>>> understand the output ;)
>>>>>>>> 
>>>>>>>> This is how I just tried it from my HBase directory:
>>>>>>>> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
>> randomRead
>>>> 10
>>>>>>>> 
>>>>>>>> And this is the output (last lines):
>>>>>>>> 10066326    56314
>>>>>>>> 10171183    56385
>>>>>>>> 10276040    56579
>>>>>>>> 10380897    56553
>>>>>>>> 
>>>>>>>> Does ask me what it means ;) I have, so far, no idea.
>>>>>>>> 
>>>>>>>> JM
>>>>>>>> 
>>>>>>>> 2013/2/1, Dalia Sobhy <dalia.mohsobhy@hotmail.com>:
>>>>>>>>> Dear Mohamed,
>>>>>>>>> 
>>>>>>>>> I checked this link:
>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation
>>>>>>>>> 
>>>>>>>>> But how to use the commands there is no clue, note that
I am
>>>> using
>>>>>> cloudera
>>>>>>>>> manager.
>>>>>>>>> 
>>>>>>>>> The other yahoo link I couldn't understand how to use
it.
>>>>>>>>> 
>>>>>>>>> So any help please??
>>>>>>>>> 
>>>>>>>>> Sent from my iPad
>>>>>>>>> 
>>>>>>>>> On Feb 1, 2013, at 2:32 PM, "Mohammad Tariq" <
>> dontariq@gmail.com
>>>>> 
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Dalia,
>>>>>>>>>> 
>>>>>>>>>>         I think the easiest way to measure the read/write
>>>>>> throughput is
>>>>>>>>>> to use "PerformanceEvaluation" tool that comes with
the
>> Hbase
>>>>>>>>>> distribution. It spawns a map-reduce job to do the
>> reads/writes
>>>> in
>>>>>>>>>> parallel. Apart from this there are several other
ways to
>>>> benchmark
>>>>>> your
>>>>>>>>>> Hbase cluster like YSCB <
>>>> https://github.com/brianfrankcooper/YCSB>.
>>>>>> You
>>>>>>>>>> might find this
>>>>>>>>>> link<
>> http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation>
>>>>>> useful
>>>>>>>>>> that talks about Hbase performance.
>>>>>>>>>> 
>>>>>>>>>> HTH
>>>>>>>>>> 
>>>>>>>>>> Warm Regards,
>>>>>>>>>> Tariq
>>>>>>>>>> https://mtariq.jux.com/
>>>>>>>>>> cloudfront.blogspot.com
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Fri, Feb 1, 2013 at 5:41 PM, Dalia Sobhy
>>>>>>>>>> <dalia.mohsobhy@hotmail.com>wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Dear all,
>>>>>>>>>>> 
>>>>>>>>>>> I want to measure the read/write throughput for
a code on a
>>>>>> cluster of
>>>>>>>>>>> 10
>>>>>>>>>>> nodes. So is there any code or way to measure
it?
>>>>>>>>>>> 
>>>>>>>>>>> I have seen in a cloudera-based presentation
that hbase
>>>> read/write
>>>>>>>>>>> throughput = millions queries per second.
>>>>>>>>>>> 
>>>>>>>>>>> So any help please??
>>>>>>>>>>> 
>>>>>>>>>>> Thanks
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Dalia Sobhy
> 
> 
> 
> -- 
> Thanks & Regards,
> Anil Gupta
Mime
View raw message