hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: How to create HTableInterface in coprocessor?
Date Tue, 22 Oct 2013 21:58:40 GMT
Yong:
There is unit test exercising CoprocessorEnvironment.getTable().
See src/test/java/org/apache/hadoop/hbase/coprocessor/TestOpenTableInCoprocessor.java
:

      HTableInterface table = e.getEnvironment().getTable(otherTable);
      Put p = new Put(new byte[] { 'a' });
      p.add(family, null, new byte[] { 'a' });
      table.put(put);

If you can modify the above test to show how the exception is reproduced,
that would help us fully understand the case and verify the fix.

Thanks


On Tue, Oct 22, 2013 at 12:51 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> I logged HBASE-9819 to backport HBASE-8372 'Provide mutability to
> CompoundConfiguration' to 0.94
>
> If you have time, you can work on the backport.
>
> Cheers
>
>
> On Tue, Oct 22, 2013 at 11:56 AM, yonghu <yongyong313@gmail.com> wrote:
>
>> Hi Ted,
>>
>> This is because I tried different ways to generate a HTableInterface.
>>
>> One is as Gray mentioned, use RegionCoprocessorEnvironment "rce" to create
>> a HTableInterface "td", but it did not work. So I commented it.
>>
>> Later I tried the approach which is suggested by
>> http://hbase.apache.org/book.html#client.connections. First create a
>> HConnection "hc", and then create a  HTableInterface "td". It still did
>> not
>> work.
>>
>> Both of them return the same error messages. Such as:
>>
>> ERROR: org.apache.hadoop.hbase.
>> client.RetriesExhaustedWithDetailsException: Failed 1 action:
>> org.apache.hadoop.hbase.DoNotRetryIOException: Coprocessor:
>>
>> 'org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment@9a99eb
>> '
>> threw: 'java.lang.UnsupportedOperationException: Immutable Configuration'
>> and has been removedfrom the active coprocessor set.
>>     at
>>
>> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.handleCoprocessorThrowable(CoprocessorHost.java:740)
>>     at
>>
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:810)
>> .............................
>>
>> and indicate the errors caused by rce.getTable() or hc.getTable().
>>
>> regards!
>>
>> Yong
>>
>>
>> On Tue, Oct 22, 2013 at 8:42 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > There're two types of exceptions. In the code below, I saw
>> rce.getTable()
>> > being commented out.
>> >
>> > Can you tell us the correlation between types of exception and
>> getTable()
>> > calls ?
>> >
>> > Thanks
>> >
>> >
>> > On Tue, Oct 22, 2013 at 11:24 AM, yonghu <yongyong313@gmail.com> wrote:
>> >
>> > > public void prePut(ObserverContext<RegionCoprocessorEnvironment>
e,
>> Put
>> > > put, WALEdit edit, boolean writeToWAL){
>> > >         RegionCoprocessorEnvironment rce = e.getEnvironment();
>> > > HTableInterface td = null;
>> > >         HTableDescriptor htd = hr.getTableDesc();
>> > >         Configuration conf = rce.getConfiguration();
>> > >         HConnection hc = null;
>> > >         try {
>> > >             hc = HConnectionManager.createConnection(conf);
>> > >         } catch (ZooKeeperConnectionException e1) {
>> > >             // TODO Auto-generated catch block
>> > >             e1.printStackTrace();
>> > >         }
>> > >         try {
>> > >             td = hc.getTable(Bytes.toBytes(tracking));
>> > >         } catch (IOException e1) {
>> > >             // TODO Auto-generated catch block
>> > >             e1.printStackTrace();
>> > >         }
>> > >         try {
>> > >             //td = rce.getTable(Bytes.toBytes(tracking));
>> > >             Put p = new Put(put.getRow());
>> > >             p.add(Bytes.toBytes("Value"), Bytes.toBytes("Current"),
>> > > Bytes.toBytes(1));
>> > >             td.put(p);
>> > >         } catch (IOException e2) {
>> > >             // TODO Auto-generated catch block
>> > >             e2.printStackTrace();
>> > >         }
>> > > }
>> > >
>> > >
>> > > On Tue, Oct 22, 2013 at 8:20 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > >
>> > > > Can you show us your code around the following line ?
>> > > >
>> > CDCTrigger.TriggerForModification.prePut(TriggerForModification.java:51)
>> > > >
>> > > > The error was due to:
>> > > >
>> > > >     public HTableInterface getTable(byte[] tableName,
>> ExecutorService
>> > > pool)
>> > > > throws IOException {
>> > > >       if (managed) {
>> > > >         throw new IOException("The connection has to be
>> unmanaged.");
>> > > >       }
>> > > >
>> > > > Cheers
>> > > >
>> > > >
>> > > > On Tue, Oct 22, 2013 at 11:14 AM, yonghu <yongyong313@gmail.com>
>> > wrote:
>> > > >
>> > > > > Ted,
>> > > > >
>> > > > > Can you tell me how to dump the stack trace of HBase? By the
way,
>> I
>> > > check
>> > > > > the log of RegionServer. It has following error messages:
>> > > > >
>> > > > > java.io.IOException: The connection has to be unmanaged.
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTable(HConnectionManager.java:669)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTable(HConnectionManager.java:658)
>> > > > >     at
>> > > > >
>> > >
>> CDCTrigger.TriggerForModification.prePut(TriggerForModification.java:51)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:808)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2196)
>> > > > >     at
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2172)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3811)
>> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > > > >     at java.lang.reflect.Method.invoke(Method.java:597)
>> > > > >     at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
>> > > > >     at
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
>> > > > >
>> > > > >
>> > > > > On Tue, Oct 22, 2013 at 8:07 PM, Ted Yu <yuzhihong@gmail.com>
>> wrote:
>> > > > >
>> > > > > > Yong:
>> > > > > > Can you post full stack trace so that we can diagnose the
>> problem ?
>> > > > > >
>> > > > > > Cheers
>> > > > > >
>> > > > > >
>> > > > > > On Tue, Oct 22, 2013 at 11:01 AM, yonghu <yongyong313@gmail.com
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > > Gray,
>> > > > > > >
>> > > > > > > Finally, I saw the error messages. ERROR:
>> > > > > > >
>> > > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>> > > > > > Failed
>> > > > > > > 1 action: org.apache.hadoop.hbase.DoNotRetryIOException:
>> > > Coprocessor:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> 'org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment@303a60
>> > > > > > > '
>> > > > > > > threw: 'java.lang.UnsupportedOperationException: Immutable
>> > > > > Configuration'
>> > > > > > > and has been removed from the active coprocessor set.
>> > > > > > >
>> > > > > > > I will try different approach as Ted mentioned.
>> > > > > > >
>> > > > > > >
>> > > > > > > On Tue, Oct 22, 2013 at 7:49 PM, yonghu <
>> yongyong313@gmail.com>
>> > > > wrote:
>> > > > > > >
>> > > > > > > > Gray
>> > > > > > > >
>> > > > > > > > Thanks for your response. I tried your approach.
But it did
>> not
>> > > > work.
>> > > > > > The
>> > > > > > > > HBase just stalled, no messages, nothing happened.
By the
>> way,
>> > my
>> > > > > hbase
>> > > > > > > > version is 0.94.12.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Tue, Oct 22, 2013 at 7:34 PM, Gary Helmling
<
>> > > > ghelmling@gmail.com
>> > > > > > > >wrote:
>> > > > > > > >
>> > > > > > > >> Within a coprocessor, you can just use the
>> > > CoprocessorEnvironment
>> > > > > > > instance
>> > > > > > > >> passed to start() method or any of the pre/post
hooks, and
>> > call
>> > > > > > > >> CoprocessorEnvironment.getTable(byte[] tablename).
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Tue, Oct 22, 2013 at 9:41 AM, Ted Yu <
>> yuzhihong@gmail.com>
>> > > > > wrote:
>> > > > > > > >>
>> > > > > > > >> > Take a look at
>> > > > > http://hbase.apache.org/book.html#client.connections,
>> > > > > > > >> > especially 9.3.1.1.
>> > > > > > > >> >
>> > > > > > > >> >
>> > > > > > > >> > On Tue, Oct 22, 2013 at 9:37 AM, yonghu
<
>> > > yongyong313@gmail.com>
>> > > > > > > wrote:
>> > > > > > > >> >
>> > > > > > > >> > > Hello,
>> > > > > > > >> > >
>> > > > > > > >> > > In the oldest verison of HBase ,
I can get the
>> > > HTableInterface
>> > > > > by
>> > > > > > > >> > > HTablePool.getTable() method. However,
in the latest
>> Hbase
>> > > > > > > >> > version0.94.12,
>> > > > > > > >> > > HTablePool is deprecated. So, I
tried to use
>> > > > HConnectionManager
>> > > > > to
>> > > > > > > >> create
>> > > > > > > >> > > HTableInterface, but it does not
work. Can anyone tell
>> me
>> > > how
>> > > > to
>> > > > > > > >> create
>> > > > > > > >> > > HTableInterface in new HBase version?
By the way,
>> there is
>> > > no
>> > > > > > error
>> > > > > > > >> > message
>> > > > > > > >> > > when I run coprocessor.
>> > > > > > > >> > >
>> > > > > > > >> > > regards!
>> > > > > > > >> > >
>> > > > > > > >> > > Yong
>> > > > > > > >> > >
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message