hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sriram Muthuswamy Chittathoor" <srir...@ivycomptech.com>
Subject RE: Support for MultiGet / SQL In clause -- error in patch HBASE-1845
Date Fri, 22 Jan 2010 20:57:06 GMT
I installed hbase-0.20.3.  

1. Can I copy my hbase data directory which had old data. 
2. I get this regionserver error --   

Fri Jan 22 11:03:33 EST 2010 Starting regionserver on
morungole.ivycomptech.co.in
ulimit -n 8192
2010-01-22 11:03:34,694 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer:
vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-Dhbase.log.dir=/home/ppoker/test/hbase-0.20.3/bin/../logs,
-Dhbase.log.file=hbase-ppoker-regionserver-morungole.ivycomptech.co.in.l
og, -Dhbase.home.dir=/home/ppoker/test/hbase-0.20.3/bin/..,
-Dhbase.id.str=ppoker, -Dhbase.root.logger=INFO,DRFA,
-Djava.library.path=/home/ppoker/test/hbase-0.20.3/bin/../lib/native/Lin
ux-i386-32]
2010-01-22 11:03:34,705 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Can not start region
server because java.lang.NoSuchMethodException:
org.apache.hadoop.hbase.regionserver.IdxRegion.<init>(org.apache.hadoop.
hbase.HBaseConfiguration)
        at java.lang.Class.getConstructor0(Class.java:2706)
        at java.lang.Class.getConstructor(Class.java:1657)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.doMain(HRegionServer.
java:2429)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.ja
va:2499)





-----Original Message-----
From: Dan Washusen [mailto:dan@reactive.org] 
Sent: Friday, January 22, 2010 4:53 AM
To: hbase-user@hadoop.apache.org
Subject: Re: Support for MultiGet / SQL In clause -- error in patch
HBASE-1845

If you want to give the "indexed" contrib package a try you'll need to
do
the following:

   1. Include the contrib jars (export HBASE_CLASSPATH=(`find
   /path/to/hbase/hbase-0.20.3/contrib/indexed -name '*jar' | tr -s "\n"
":"`)
   2. Set the 'hbase.hregion.impl' property to
   'org.apache.hadoop.hbase.regionserver.IdxRegion' in your
hbase-site.xml

Once you've done that you can create a table with an index using:

>     // define which qualifiers need an index (choosing the correct
type)
>     IdxColumnDescriptor columnDescriptor = new
> IdxColumnDescriptor("columnFamily");
>     columnDescriptor.addIndexDescriptor(
>       new IdxIndexDescriptor("qualifier", IdxQualifierType.BYTE_ARRAY)
>     );
>
>     HTableDescriptor tableDescriptor = new HTableDescriptor("table");
>     tableDescriptor.addFamily(columnDescriptor);


Then when you want to perform a scan with an index hint:

>     Scan scan = new IdxScan(
>           new Comparison("columnFamily", "qualifier",
>               Comparison.Operator.EQ, Bytes.toBytes("foo"))
>       );
>

You have to keep in mind that the index hint is only a hint.  It
guarantees
that your scan will get all rows that match the hint but you'll more
than
likely receive rows that don't.  For this reason I'd suggest that you
also
include a filter along with the scan:

>       Scan scan = new IdxScan(
>           new Comparison("columnFamily", "qualifier",
>               Comparison.Operator.EQ, Bytes.toBytes("foo"))
>       );
>       scan.setFilter(
>           new SingleColumnValueFilter(
>               "columnFamily", "qualifer",
CompareFilter.CompareOp.EQUAL,
>               new BinaryComparator("foo")
>           )
>       );
>

Cheers,
Dan


2010/1/22 stack <stack@duboce.net>

>
http://people.apache.org/~jdcryans/hbase-0.20.3-candidate-2/<http://peop
le.apache.org/%7Ejdcryans/hbase-0.20.3-candidate-2/>
>
> There is a bit of documentation if you look at javadoc for the
> 'indexed' contrib (This is what hbase-2073 is called on commit).
>
> St.Ack
>
> P.S. We had a thread going named "HBase bulk load".  You got all the
> answers you need on that one?
>
> On Thu, Jan 21, 2010 at 11:19 AM, Sriram Muthuswamy Chittathoor
> <sriramc@ivycomptech.com> wrote:
> >
> > Great.  Can I migrate to 0.20.3RC2 easily.  I am on 0.20.2. Can u
pass
> > me the link
> >
> > -----Original Message-----
> > From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> > stack
> > Sent: Friday, January 22, 2010 12:42 AM
> > To: hbase-user@hadoop.apache.org
> > Subject: Re: Support for MultiGet / SQL In clause -- error in patch
> > HBASE-1845
> >
> > IIRC, hbase-1845 was a sketch only and not yet complete.  Its
probably
> > rotted since any ways.
> >
> > Have you looked at hbase-2037 since committed and available in
> > 0.20.3RC2.
> >  Would this help you with your original problem?
> >
> > St.Ack
> >
> > On Thu, Jan 21, 2010 at 9:10 AM, Sriram Muthuswamy Chittathoor <
> > sriramc@ivycomptech.com> wrote:
> >
> > > I tried applying the patch to the hbase source code  hbase 0.20.2
and
> > I
> > > get the errors below.  Do you know if this needs to be applied to
a
> > > specific hbase version. Is there a version which works with 0.20.2
or
> > > later ??
> > > Basically HRegionServer  and HTable patching fails.
> > >
> > >
> > > Thanks for the help
> > >
> > > patch -p0 -i batch.patch
> > >
> > > patching file src/java/org/apache/hadoop/hbase/client/Get.java
> > > Hunk #1 succeeded at 61 (offset 2 lines).
> > > Hunk #2 succeeded at 347 (offset 31 lines).
> > > patching file
src/java/org/apache/hadoop/hbase/client/HConnection.java
> > > patching file
> > > src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
> > > Hunk #3 succeeded at 1244 (offset 6 lines).
> > > patching file src/java/org/apache/hadoop/hbase/client/HTable.java
> > > Hunk #2 succeeded at 73 (offset 8 lines).
> > > Hunk #4 FAILED at 405.
> > > Hunk #5 succeeded at 671 with fuzz 2 (offset 26 lines).
> > > 1 out of 5 hunks FAILED -- saving rejects to file
> > > src/java/org/apache/hadoop/hbase/client/HTable.java.rej
> > > patching file src/java/org/apache/hadoop/hbase/client/Multi.java
> > > patching file
> > src/java/org/apache/hadoop/hbase/client/MultiCallable.java
> > > patching file
src/java/org/apache/hadoop/hbase/client/MultiResult.java
> > > patching file src/java/org/apache/hadoop/hbase/client/Row.java
> > > patching file
> > > src/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
> > > Hunk #2 succeeded at 156 with fuzz 1 (offset 3 lines).
> > > patching file
> > src/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
> > > Hunk #2 succeeded at 247 (offset 2 lines).
> > > patching file
> > > src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> > > Hunk #1 succeeded at 78 (offset -1 lines).
> > > Hunk #2 FAILED at 2515.
> > > 1 out of 2 hunks FAILED -- saving rejects to file
> > >
src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java.rej
> > > patching file
src/test/org/apache/hadoop/hbase/client/TestHTable.java
> > > Hunk #2 FAILED at 333.
> > > 1 out of 2 hunks FAILED -- saving rejects to file
> > > src/test/org/apache/hadoop/hbase/client/TestHTable.java.rej
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Marc Limotte [mailto:mslimotte@gmail.com]
> > > Sent: Tuesday, January 19, 2010 10:26 PM
> > > To: hbase-user@hadoop.apache.org
> > > Subject: Re: Support for MultiGet / SQL In clause
> > >
> > > Sriram,
> > >
> > > Would a secondary index help you:
> > >
> >
http://hadoop.apache.org/hbase/docs/r0.20.2/api/org/apache/hadoop/hbase/
> > > client/tableindexed/package-summary.html#package_description
> > > .
> > >
> > > The index is stored in a separate table, but the index is managed
for
> > > you.
> > >
> > > I don't think you can do an arbitrary "in" query, though.  If the
keys
> > > that
> > > you want to include in the "in" are reasonably close neighbors,
you
> > > could do
> > > a scan and skip ones that are uninteresting.  You could also try a
> > batch
> > > Get
> > > by applying a separate patch, see
> > > http://issues.apache.org/jira/browse/HBASE-1845.
> > >
> > > Marc Limotte
> > >
> > > On Tue, Jan 19, 2010 at 8:45 AM, Sriram Muthuswamy Chittathoor <
> > > sriramc@ivycomptech.com> wrote:
> > >
> > > > Is there any support for this.  I want to do this
> > > >
> > > > 1.  Create a second table to maintain mapping between secondary
> > column
> > > > and the rowid's of the primary table
> > > >
> > > > 2.  Use this second table to get the rowid's to lookup from the
> > > primary
> > > > table using a SQL In like clause ---
> > > >
> > > > Basically I am doing this to speed up querying by  Non-row key
> > > columns.
> > > >
> > > > Thanks
> > > >
> > > > Sriram C
> > > >
> > > >
> > > > This email is sent for and on behalf of Ivy Comptech Private
> > Limited.
> > > Ivy
> > > > Comptech Private Limited is a limited liability company.
> > > >
> > > > This email and any attachments are confidential, and may be
legally
> > > > privileged and protected by copyright. If you are not the
intended
> > > recipient
> > > > dissemination or copying of this email is prohibited. If you
have
> > > received
> > > > this in error, please notify the sender by replying by email and
> > then
> > > delete
> > > > the email completely from your system.
> > > > Any views or opinions are solely those of the sender.  This
> > > communication
> > > > is not intended to form a binding contract on behalf of Ivy
Comptech
> > > Private
> > > > Limited unless expressly indicated to the contrary and properly
> > > authorised.
> > > > Any actions taken on the basis of this email are at the
recipient's
> > > own
> > > > risk.
> > > >
> > > > Registered office:
> > > > Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara
> > Hills,
> > > > Hyderabad 500 033, Andhra Pradesh, India. Registered number:
37994.
> > > > Registered in India. A list of members' names is available for
> > > inspection at
> > > > the registered office.
> > > >
> > > >
> > >
>

Mime
View raw message