hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Washusen <...@reactive.org>
Subject Re: Support for MultiGet / SQL In clause -- error in patch HBASE-1845
Date Fri, 22 Jan 2010 22:06:45 GMT
This error looks like you might be setting the 'hbase.regionserver.impl'
property to "org.apache.hadoop.hbase.regionserver.IdxRegion" instead of the
'hbase.hregion.impl' property.  Could you ensure that you have the following
in your hbase-site.xml:

...

<property>
>     <name>hbase.hregion.impl</name>
>     <value>org.apache.hadoop.hbase.regionserver.IdxRegion</value>
> </property>
> ...
>


2010/1/23 Sriram Muthuswamy Chittathoor <sriramc@ivycomptech.com>

> I installed hbase-0.20.3.
>
> 1. Can I copy my hbase data directory which had old data.
> 2. I get this regionserver error --
>
> Fri Jan 22 11:03:33 EST 2010 Starting regionserver on
> morungole.ivycomptech.co.in
> ulimit -n 8192
> 2010-01-22 11:03:34,694 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
> -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
> -Dhbase.log.dir=/home/ppoker/test/hbase-0.20.3/bin/../logs,
> -Dhbase.log.file=hbase-ppoker-regionserver-morungole.ivycomptech.co.in.l
> og, -Dhbase.home.dir=/home/ppoker/test/hbase-0.20.3/bin/..,
> -Dhbase.id.str=ppoker, -Dhbase.root.logger=INFO,DRFA,
> -Djava.library.path=/home/ppoker/test/hbase-0.20.3/bin/../lib/native/Lin
> ux-i386-32]
> 2010-01-22 11:03:34,705 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Can not start region
> server because java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.regionserver.IdxRegion.<init>(org.apache.hadoop.
> hbase.HBaseConfiguration)
>        at java.lang.Class.getConstructor0(Class.java:2706)
>        at java.lang.Class.getConstructor(Class.java:1657)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doMain(HRegionServer.
> java:2429)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.ja
> va:2499)
>
>
>
>
>
> -----Original Message-----
> From: Dan Washusen [mailto:dan@reactive.org]
> Sent: Friday, January 22, 2010 4:53 AM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Support for MultiGet / SQL In clause -- error in patch
> HBASE-1845
>
> If you want to give the "indexed" contrib package a try you'll need to
> do
> the following:
>
>    1. Include the contrib jars (export HBASE_CLASSPATH=(`find
>    /path/to/hbase/hbase-0.20.3/contrib/indexed -name '*jar' | tr -s "\n"
> ":"`)
>    2. Set the 'hbase.hregion.impl' property to
>    'org.apache.hadoop.hbase.regionserver.IdxRegion' in your
> hbase-site.xml
>
> Once you've done that you can create a table with an index using:
>
> >     // define which qualifiers need an index (choosing the correct
> type)
> >     IdxColumnDescriptor columnDescriptor = new
> > IdxColumnDescriptor("columnFamily");
> >     columnDescriptor.addIndexDescriptor(
> >       new IdxIndexDescriptor("qualifier", IdxQualifierType.BYTE_ARRAY)
> >     );
> >
> >     HTableDescriptor tableDescriptor = new HTableDescriptor("table");
> >     tableDescriptor.addFamily(columnDescriptor);
>
>
> Then when you want to perform a scan with an index hint:
>
> >     Scan scan = new IdxScan(
> >           new Comparison("columnFamily", "qualifier",
> >               Comparison.Operator.EQ, Bytes.toBytes("foo"))
> >       );
> >
>
> You have to keep in mind that the index hint is only a hint.  It
> guarantees
> that your scan will get all rows that match the hint but you'll more
> than
> likely receive rows that don't.  For this reason I'd suggest that you
> also
> include a filter along with the scan:
>
> >       Scan scan = new IdxScan(
> >           new Comparison("columnFamily", "qualifier",
> >               Comparison.Operator.EQ, Bytes.toBytes("foo"))
> >       );
> >       scan.setFilter(
> >           new SingleColumnValueFilter(
> >               "columnFamily", "qualifer",
> CompareFilter.CompareOp.EQUAL,
> >               new BinaryComparator("foo")
> >           )
> >       );
> >
>
> Cheers,
> Dan
>
>
> 2010/1/22 stack <stack@duboce.net>
>
> >
> http://people.apache.org/~jdcryans/hbase-0.20.3-candidate-2/<http://people.apache.org/%7Ejdcryans/hbase-0.20.3-candidate-2/>
> <http://peop
> le.apache.org/%7Ejdcryans/hbase-0.20.3-candidate-2/>
> >
> > There is a bit of documentation if you look at javadoc for the
> > 'indexed' contrib (This is what hbase-2073 is called on commit).
> >
> > St.Ack
> >
> > P.S. We had a thread going named "HBase bulk load".  You got all the
> > answers you need on that one?
> >
> > On Thu, Jan 21, 2010 at 11:19 AM, Sriram Muthuswamy Chittathoor
> > <sriramc@ivycomptech.com> wrote:
> > >
> > > Great.  Can I migrate to 0.20.3RC2 easily.  I am on 0.20.2. Can u
> pass
> > > me the link
> > >
> > > -----Original Message-----
> > > From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> > > stack
> > > Sent: Friday, January 22, 2010 12:42 AM
> > > To: hbase-user@hadoop.apache.org
> > > Subject: Re: Support for MultiGet / SQL In clause -- error in patch
> > > HBASE-1845
> > >
> > > IIRC, hbase-1845 was a sketch only and not yet complete.  Its
> probably
> > > rotted since any ways.
> > >
> > > Have you looked at hbase-2037 since committed and available in
> > > 0.20.3RC2.
> > >  Would this help you with your original problem?
> > >
> > > St.Ack
> > >
> > > On Thu, Jan 21, 2010 at 9:10 AM, Sriram Muthuswamy Chittathoor <
> > > sriramc@ivycomptech.com> wrote:
> > >
> > > > I tried applying the patch to the hbase source code  hbase 0.20.2
> and
> > > I
> > > > get the errors below.  Do you know if this needs to be applied to
> a
> > > > specific hbase version. Is there a version which works with 0.20.2
> or
> > > > later ??
> > > > Basically HRegionServer  and HTable patching fails.
> > > >
> > > >
> > > > Thanks for the help
> > > >
> > > > patch -p0 -i batch.patch
> > > >
> > > > patching file src/java/org/apache/hadoop/hbase/client/Get.java
> > > > Hunk #1 succeeded at 61 (offset 2 lines).
> > > > Hunk #2 succeeded at 347 (offset 31 lines).
> > > > patching file
> src/java/org/apache/hadoop/hbase/client/HConnection.java
> > > > patching file
> > > > src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
> > > > Hunk #3 succeeded at 1244 (offset 6 lines).
> > > > patching file src/java/org/apache/hadoop/hbase/client/HTable.java
> > > > Hunk #2 succeeded at 73 (offset 8 lines).
> > > > Hunk #4 FAILED at 405.
> > > > Hunk #5 succeeded at 671 with fuzz 2 (offset 26 lines).
> > > > 1 out of 5 hunks FAILED -- saving rejects to file
> > > > src/java/org/apache/hadoop/hbase/client/HTable.java.rej
> > > > patching file src/java/org/apache/hadoop/hbase/client/Multi.java
> > > > patching file
> > > src/java/org/apache/hadoop/hbase/client/MultiCallable.java
> > > > patching file
> src/java/org/apache/hadoop/hbase/client/MultiResult.java
> > > > patching file src/java/org/apache/hadoop/hbase/client/Row.java
> > > > patching file
> > > > src/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
> > > > Hunk #2 succeeded at 156 with fuzz 1 (offset 3 lines).
> > > > patching file
> > > src/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
> > > > Hunk #2 succeeded at 247 (offset 2 lines).
> > > > patching file
> > > > src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> > > > Hunk #1 succeeded at 78 (offset -1 lines).
> > > > Hunk #2 FAILED at 2515.
> > > > 1 out of 2 hunks FAILED -- saving rejects to file
> > > >
> src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java.rej
> > > > patching file
> src/test/org/apache/hadoop/hbase/client/TestHTable.java
> > > > Hunk #2 FAILED at 333.
> > > > 1 out of 2 hunks FAILED -- saving rejects to file
> > > > src/test/org/apache/hadoop/hbase/client/TestHTable.java.rej
> > > >
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Marc Limotte [mailto:mslimotte@gmail.com]
> > > > Sent: Tuesday, January 19, 2010 10:26 PM
> > > > To: hbase-user@hadoop.apache.org
> > > > Subject: Re: Support for MultiGet / SQL In clause
> > > >
> > > > Sriram,
> > > >
> > > > Would a secondary index help you:
> > > >
> > >
> http://hadoop.apache.org/hbase/docs/r0.20.2/api/org/apache/hadoop/hbase/
> > > > client/tableindexed/package-summary.html#package_description
> > > > .
> > > >
> > > > The index is stored in a separate table, but the index is managed
> for
> > > > you.
> > > >
> > > > I don't think you can do an arbitrary "in" query, though.  If the
> keys
> > > > that
> > > > you want to include in the "in" are reasonably close neighbors,
> you
> > > > could do
> > > > a scan and skip ones that are uninteresting.  You could also try a
> > > batch
> > > > Get
> > > > by applying a separate patch, see
> > > > http://issues.apache.org/jira/browse/HBASE-1845.
> > > >
> > > > Marc Limotte
> > > >
> > > > On Tue, Jan 19, 2010 at 8:45 AM, Sriram Muthuswamy Chittathoor <
> > > > sriramc@ivycomptech.com> wrote:
> > > >
> > > > > Is there any support for this.  I want to do this
> > > > >
> > > > > 1.  Create a second table to maintain mapping between secondary
> > > column
> > > > > and the rowid's of the primary table
> > > > >
> > > > > 2.  Use this second table to get the rowid's to lookup from the
> > > > primary
> > > > > table using a SQL In like clause ---
> > > > >
> > > > > Basically I am doing this to speed up querying by  Non-row key
> > > > columns.
> > > > >
> > > > > Thanks
> > > > >
> > > > > Sriram C
> > > > >
> > > > >
> > > > > This email is sent for and on behalf of Ivy Comptech Private
> > > Limited.
> > > > Ivy
> > > > > Comptech Private Limited is a limited liability company.
> > > > >
> > > > > This email and any attachments are confidential, and may be
> legally
> > > > > privileged and protected by copyright. If you are not the
> intended
> > > > recipient
> > > > > dissemination or copying of this email is prohibited. If you
> have
> > > > received
> > > > > this in error, please notify the sender by replying by email and
> > > then
> > > > delete
> > > > > the email completely from your system.
> > > > > Any views or opinions are solely those of the sender.  This
> > > > communication
> > > > > is not intended to form a binding contract on behalf of Ivy
> Comptech
> > > > Private
> > > > > Limited unless expressly indicated to the contrary and properly
> > > > authorised.
> > > > > Any actions taken on the basis of this email are at the
> recipient's
> > > > own
> > > > > risk.
> > > > >
> > > > > Registered office:
> > > > > Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara
> > > Hills,
> > > > > Hyderabad 500 033, Andhra Pradesh, India. Registered number:
> 37994.
> > > > > Registered in India. A list of members' names is available for
> > > > inspection at
> > > > > the registered office.
> > > > >
> > > > >
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message