hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: YCSB on hbase fails with java.io.EOFException
Date Fri, 17 May 2013 03:11:34 GMT
Here is the hbase-site.xml I used. ycsb1 is the name of the vm. I have
simply updated the hbase-site.xml provided with ycsb.

$ cat hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 * Copyright 2009 The Apache Software Foundation
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://ycsb1:20001/hbase</value>
    <description>The directory shared by region servers.
    </description>
  </property>
  <property>
    <name>hbase.master</name>
    <value>ycsb1:60000</value>
    <description>The host and port that the HBase master runs at.
    </description>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>ycsb1</value>
  </property>
</configuration>



2013/5/16 Manoj S <manoj.sundaram21@gmail.com>

> Tried your way too. Same results again:( Let me try installing it on a VM.
>
> Also, if you don't mind can you share me the contents you added to YCSB's
> hbase-site.xml?
> Thanks
> On Fri, May 17, 2013 at 7:38 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Hum.
>>
>> I did slightly different steps.
>>
>> 1) I try on standalone. I wlil try on fully distributed soon.
>> 2) I did not untared anything. After maven compilation, I used the ycsb
>> file in bin.
>> 3) Simply changed the required lines in the hbase-site.xml
>> 4) created the table with the shell, only one single region create
>> 'usertable', { NAME => 'cf' }
>> 5) did not tried with recordcount neither thrads.
>>
>> So 3 things.
>>
>> First, can you try with the ycsb file directly in bin without "untaring"
>> anything?
>> Second, can you update the existing hbase-site.xml instead of overwriting
>> it with your HBase one?
>> Third, can you try without recordCount and threads?
>>
>> On my side I will try to find more instances to install 0.94.1 and
>> retry...
>>
>> JMS
>>
>>
>> 2013/5/16 Manoj S <manoj.sundaram21@gmail.com>
>>
>>> Jean,
>>>
>>> That's interesting. I downloaded the ycsb source from github(
>>> https://github.com/brianfrankcooper/YCSB) and compiled the code myself.
>>> ( Installation from the tar ball did not work either. Faced a different
>>> issue. Building from the source code solved that problem for me ).
>>>
>>> I'm listing down the steps I performed.
>>> 1) On a perfectly working Hadoop(Intel distro)/Hbase(0.94.1) cluster, I
>>> downloaded ycsb-0.1.4 from git hub and compiled it with maven.
>>> 2) Untar'd the new tar ball.
>>> 3) Copied my hbase-site.xml to
>>> "ycsb-0.1.4/hbase-binding/conf/hbase-site.xml".
>>> 4) Created the table in hbase with "hbase
>>> org.apache.hadoop.hbase.util.RegionSplitter usertable UniformSplit -c 200
>>> -f family"
>>> 5) ycsb-0.1.4/bin/ycsb load hbase -P workloads/workloada -p
>>> columnfamily=family -p recordcount=10  -threads 3 -s > data.tmp
>>>
>>> Step 5 threw me the error. Can you tell me if i have missed something
>>> here?
>>>
>>> Thanks,
>>> Manoj
>>>
>>> On May 17, 2013 9:32 AM, "Jean-Marc Spaggiari" <jean-marc@spaggiari.org>
>>> wrote:
>>> >
>>> > Hi Manoj,
>>> >
>>> > I installed 0.94.1 and YCSB and everything is working fine.
>>> >
>>> > Which YCSB version are you using? Can you try to export the one from
>>> GitHub?
>>> >
>>> > JM
>>> >
>>> >
>>> > 2013/5/15 Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>>> >>
>>> >> No worries. I will prepare a VM and install 0.94.1 on it.
>>> >>
>>> >> I will let you know when it's done.
>>> >>
>>> >>
>>> >> JM
>>> >>
>>> >> 2013/5/15 Manoj S <manoj.sundaram21@gmail.com>
>>> >>>
>>> >>> Jean,
>>> >>>
>>> >>> Unfortunately, I will not be able to install a newer version of
>>> HBase and test :(
>>> >>>
>>> >>> It is a production environment.
>>> >>>
>>> >>> Appreciate your help!
>>> >>>
>>> >>> -Manoj
>>> >>>
>>> >>> On May 16, 2013 8:46 AM, "Jean-Marc Spaggiari" <
>>> jean-marc@spaggiari.org> wrote:
>>> >>> >
>>> >>> > Hi Manoj,
>>> >>> >
>>> >>> > Is it a production environment? Or a test environment? If it's
a
>>> test one, can you try with a recent versoin like 0.94.7?
>>> >>> >
>>> >>> > Else I will try to instal 0.94.1 locally...
>>> >>> >
>>> >>> > JM
>>> >>> >
>>> >>> >
>>> >>> > 2013/5/15 Manoj S <manoj.sundaram21@gmail.com>
>>> >>> >>
>>> >>> >> Hi Jean,
>>> >>> >>
>>> >>> >> I am using 0.94.1
>>> >>> >>
>>> >>> >> -Manoj
>>> >>> >>
>>> >>> >> On May 15, 2013 8:05 PM, "Jean-Marc Spaggiari" <
>>> jean-marc@spaggiari.org> wrote:
>>> >>> >> >
>>> >>> >> > Regarding the version, which 0.94 are you using? 0.94.1?
Or
>>> 0.94.7? I will install it locally and give it a try...
>>> >>> >> >
>>> >>> >> > 2013/5/15 S, Manoj <manoj.s@intel.com>
>>> >>> >> >
>>> >>> >> >> Hi Jean,
>>> >>> >> >>
>>> >>> >> >> Yeah. All these operations work fine.
>>> >>> >> >>
>>> >>> >> >> Thanks,
>>> >>> >> >> Manoj
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >> -----Original Message-----
>>> >>> >> >> From: Jean-Marc Spaggiari [jean-marc@spaggiari.org]
>>> >>> >> >> Sent: Wednesday, May 15, 2013 07:58 PM Malay Peninsula
>>> Standard Time
>>> >>> >> >> To: user@hadoop.apache.org
>>> >>> >> >> Subject: Re: YCSB on hbase fails with java.io.EOFException
>>> >>> >> >>
>>> >>> >> >> Hi Manoj,
>>> >>> >> >>
>>> >>> >> >> Are you able to do basic operations with HBase
shell? Like
>>> scan, put, get, etc.?
>>> >>> >> >>
>>> >>> >> >> WebUI is show tables correctly too?
>>> >>> >> >>
>>> >>> >> >> 2013/5/15 Manoj S <manoj.sundaram21@gmail.com>
>>> >>> >> >>>
>>> >>> >> >>> Yes, it is running fine.
>>> >>> >> >>>
>>> >>> >> >>> I am using version 0.94.
>>> >>> >> >>>
>>> >>> >> >>> Thanks,
>>> >>> >> >>> Manoj
>>> >>> >> >>>
>>> >>> >> >>> On Wed, May 15, 2013 at 10:14 AM, Ted Yu <yuzhihong@gmail.com>
>>> wrote:
>>> >>> >> >>>>
>>> >>> >> >>>> I assume region server was running fine
>>> on <hostname>/<IP>:60020
>>> >>> >> >>>>
>>> >>> >> >>>> BTW what HBase version are you using ?
>>> >>> >> >>>>
>>> >>> >> >>>> Thanks
>>> >>> >> >>>>
>>> >>> >> >>>> On Tue, May 14, 2013 at 8:49 PM, Manoj
S <
>>> manoj.sundaram21@gmail.com> wrote:
>>> >>> >> >>>>>
>>> >>> >> >>>>> Hi,
>>> >>> >> >>>>>
>>> >>> >> >>>>> I am trying to benchmark hbase with
YCSB.
>>> >>> >> >>>>>
>>> >>> >> >>>>> #  ./bin/ycsb load hbase -p columnfamily=family
-P
>>> workloads/workloada > /tmp/loaddata.tmp
>>> >>> >> >>>>>
>>> >>> >> >>>>> Loading workload...
>>> >>> >> >>>>>
>>> >>> >> >>>>> Starting test.
>>> >>> >> >>>>>
>>> >>> >> >>>>> SLF4J: The requested version 1.5.8
by your slf4j binding is
>>> not compatible with [1.6]
>>> >>> >> >>>>>
>>> >>> >> >>>>> SLF4J: See http://www.slf4j.org/codes.html#version_mismatchfor
further details.
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:host.name=<hostname>
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.version=1.6.0_24
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.vendor=Sun Microsystems Inc.
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.class.path=/<path>/ycsb-0.1.4/hbase-binding/conf:/<path>/ycsb-0.1.4/hbase-binding/lib/hbase-binding-0.1.4.jar:/<path>/ycsb-0.1.4/core/lib/core-0.1.4.jar:/<path>/ycsb-0.1.4/voldemort-binding/conf:/<path>/ycsb-0.1.4/jdbc-binding/conf:/<path>/ycsb-0.1.4/infinispan-binding/conf
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.io.tmpdir=/tmp
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:java.compiler=<NA>
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:os.name=Linux
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:os.arch=amd64
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:os.version=2.6.32-279.el6.x86_64
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:user.name=hbase
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:user.home=/var/run/hbase
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Client
>>> environment:user.dir=/<path>/ycsb-0.1.4
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ZooKeeper:
Initiating
>>> client connection, connectString=<hostname>:2181 sessionTimeout=180000
>>> watcher=hconnection
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.RecoverableZooKeeper:
The
>>> identifier of this process is 24314@<hostname>
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ClientCnxn:
Opening socket
>>> connection to server /<IP>:2181
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO client.ZooKeeperSaslClient:
Client
>>> will not SASL-authenticate because the default JAAS configuration section
>>> 'Client' could not be found. If you are not using SASL, you may ignore
>>> this. On the other hand, if you expected SASL to work, please fix your JAAS
>>> configuration.
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ClientCnxn:
Socket
>>> connection established to <hostname>/<IP>:2181, initiating session
>>> >>> >> >>>>>
>>> >>> >> >>>>> 13/05/15 11:31:50 INFO zookeeper.ClientCnxn:
Session
>>> establishment complete on server <hostname>/<IP>:2181, sessionid
=
>>> 0x3ea192c11c1bff, negotiated timeout = 180000
>>> >>> >> >>>>>
>>> >>> >> >>>>>  Error accessing HBase table: java.io.IOException:
Call to
>>> <hostname>/<IP>:60020 failed on local exception: java.io.EOFException
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>> HBase+Zookeeper are running fine and
i am able to perform
>>> other operations on it. The problem arises only when i use YCSB.
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>> Some background info:
>>> >>> >> >>>>>
>>> >>> >> >>>>> # netstat -ntpl | grep 60020
>>> >>> >> >>>>>
>>> >>> >> >>>>> tcp        0      0 ::ffff: <IP>:60020
 :::*
>>>                    LISTEN      115262/java
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>> # created user table with,
>>> >>> >> >>>>>
>>> >>> >> >>>>> # hbase org.apache.hadoop.hbase.util.RegionSplitter
>>> usertable UniformSplit -c 200 -f family
>>> >>> >> >>>>>
>>> >>> >> >>>>> Hbase related logs doesn't reveal
anything.
>>> >>> >> >>>>>
>>> >>> >> >>>>>
>>> >>> >> >>>>> Can someone point me in the right
direction?
>>> >>> >> >>>>>
>>> >>> >> >>>>> Thanks in advance for the help!
>>> >>> >> >>>>>
>>> >>> >> >>>>> -Manoj
>>> >>> >> >>>>
>>> >>> >> >>>>
>>> >>> >> >>>
>>> >>> >> >>
>>> >>> >> >
>>> >>> >
>>> >>> >
>>> >>
>>> >>
>>> >
>>>
>>
>>
>

Mime
View raw message