hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: How to create HTableInterface in coprocessor?
Date Wed, 23 Oct 2013 21:53:34 GMT
Yong:
I have attached the backport to HBASE-9819.

If you can patch your build and see if it fixes the problem, that would be
great.


On Tue, Oct 22, 2013 at 2:58 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Yong:
> There is unit test exercising CoprocessorEnvironment.getTable().
> See src/test/java/org/apache/hadoop/hbase/coprocessor/TestOpenTableInCoprocessor.java
> :
>
>       HTableInterface table = e.getEnvironment().getTable(otherTable);
>       Put p = new Put(new byte[] { 'a' });
>       p.add(family, null, new byte[] { 'a' });
>       table.put(put);
>
> If you can modify the above test to show how the exception is reproduced,
> that would help us fully understand the case and verify the fix.
>
> Thanks
>
>
> On Tue, Oct 22, 2013 at 12:51 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> I logged HBASE-9819 to backport HBASE-8372 'Provide mutability to
>> CompoundConfiguration' to 0.94
>>
>> If you have time, you can work on the backport.
>>
>> Cheers
>>
>>
>> On Tue, Oct 22, 2013 at 11:56 AM, yonghu <yongyong313@gmail.com> wrote:
>>
>>> Hi Ted,
>>>
>>> This is because I tried different ways to generate a HTableInterface.
>>>
>>> One is as Gray mentioned, use RegionCoprocessorEnvironment "rce" to
>>> create
>>> a HTableInterface "td", but it did not work. So I commented it.
>>>
>>> Later I tried the approach which is suggested by
>>> http://hbase.apache.org/book.html#client.connections. First create a
>>> HConnection "hc", and then create a  HTableInterface "td". It still did
>>> not
>>> work.
>>>
>>> Both of them return the same error messages. Such as:
>>>
>>> ERROR: org.apache.hadoop.hbase.
>>> client.RetriesExhaustedWithDetailsException: Failed 1 action:
>>> org.apache.hadoop.hbase.DoNotRetryIOException: Coprocessor:
>>>
>>> 'org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment@9a99eb
>>> '
>>> threw: 'java.lang.UnsupportedOperationException: Immutable Configuration'
>>> and has been removedfrom the active coprocessor set.
>>>     at
>>>
>>> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.handleCoprocessorThrowable(CoprocessorHost.java:740)
>>>     at
>>>
>>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:810)
>>> .............................
>>>
>>> and indicate the errors caused by rce.getTable() or hc.getTable().
>>>
>>> regards!
>>>
>>> Yong
>>>
>>>
>>> On Tue, Oct 22, 2013 at 8:42 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>
>>> > There're two types of exceptions. In the code below, I saw
>>> rce.getTable()
>>> > being commented out.
>>> >
>>> > Can you tell us the correlation between types of exception and
>>> getTable()
>>> > calls ?
>>> >
>>> > Thanks
>>> >
>>> >
>>> > On Tue, Oct 22, 2013 at 11:24 AM, yonghu <yongyong313@gmail.com>
>>> wrote:
>>> >
>>> > > public void prePut(ObserverContext<RegionCoprocessorEnvironment>
e,
>>> Put
>>> > > put, WALEdit edit, boolean writeToWAL){
>>> > >         RegionCoprocessorEnvironment rce = e.getEnvironment();
>>> > > HTableInterface td = null;
>>> > >         HTableDescriptor htd = hr.getTableDesc();
>>> > >         Configuration conf = rce.getConfiguration();
>>> > >         HConnection hc = null;
>>> > >         try {
>>> > >             hc = HConnectionManager.createConnection(conf);
>>> > >         } catch (ZooKeeperConnectionException e1) {
>>> > >             // TODO Auto-generated catch block
>>> > >             e1.printStackTrace();
>>> > >         }
>>> > >         try {
>>> > >             td = hc.getTable(Bytes.toBytes(tracking));
>>> > >         } catch (IOException e1) {
>>> > >             // TODO Auto-generated catch block
>>> > >             e1.printStackTrace();
>>> > >         }
>>> > >         try {
>>> > >             //td = rce.getTable(Bytes.toBytes(tracking));
>>> > >             Put p = new Put(put.getRow());
>>> > >             p.add(Bytes.toBytes("Value"), Bytes.toBytes("Current"),
>>> > > Bytes.toBytes(1));
>>> > >             td.put(p);
>>> > >         } catch (IOException e2) {
>>> > >             // TODO Auto-generated catch block
>>> > >             e2.printStackTrace();
>>> > >         }
>>> > > }
>>> > >
>>> > >
>>> > > On Tue, Oct 22, 2013 at 8:20 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>>> > >
>>> > > > Can you show us your code around the following line ?
>>> > > >
>>> >
>>> CDCTrigger.TriggerForModification.prePut(TriggerForModification.java:51)
>>> > > >
>>> > > > The error was due to:
>>> > > >
>>> > > >     public HTableInterface getTable(byte[] tableName,
>>> ExecutorService
>>> > > pool)
>>> > > > throws IOException {
>>> > > >       if (managed) {
>>> > > >         throw new IOException("The connection has to be
>>> unmanaged.");
>>> > > >       }
>>> > > >
>>> > > > Cheers
>>> > > >
>>> > > >
>>> > > > On Tue, Oct 22, 2013 at 11:14 AM, yonghu <yongyong313@gmail.com>
>>> > wrote:
>>> > > >
>>> > > > > Ted,
>>> > > > >
>>> > > > > Can you tell me how to dump the stack trace of HBase? By
the
>>> way, I
>>> > > check
>>> > > > > the log of RegionServer. It has following error messages:
>>> > > > >
>>> > > > > java.io.IOException: The connection has to be unmanaged.
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTable(HConnectionManager.java:669)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTable(HConnectionManager.java:658)
>>> > > > >     at
>>> > > > >
>>> > >
>>> CDCTrigger.TriggerForModification.prePut(TriggerForModification.java:51)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:808)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2196)
>>> > > > >     at
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2172)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3811)
>>> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > > > >     at java.lang.reflect.Method.invoke(Method.java:597)
>>> > > > >     at
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
>>> > > > >     at
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
>>> > > > >
>>> > > > >
>>> > > > > On Tue, Oct 22, 2013 at 8:07 PM, Ted Yu <yuzhihong@gmail.com>
>>> wrote:
>>> > > > >
>>> > > > > > Yong:
>>> > > > > > Can you post full stack trace so that we can diagnose
the
>>> problem ?
>>> > > > > >
>>> > > > > > Cheers
>>> > > > > >
>>> > > > > >
>>> > > > > > On Tue, Oct 22, 2013 at 11:01 AM, yonghu <
>>> yongyong313@gmail.com>
>>> > > > wrote:
>>> > > > > >
>>> > > > > > > Gray,
>>> > > > > > >
>>> > > > > > > Finally, I saw the error messages. ERROR:
>>> > > > > > >
>>> > > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>> > > > > > Failed
>>> > > > > > > 1 action: org.apache.hadoop.hbase.DoNotRetryIOException:
>>> > > Coprocessor:
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> 'org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment@303a60
>>> > > > > > > '
>>> > > > > > > threw: 'java.lang.UnsupportedOperationException:
Immutable
>>> > > > > Configuration'
>>> > > > > > > and has been removed from the active coprocessor
set.
>>> > > > > > >
>>> > > > > > > I will try different approach as Ted mentioned.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > On Tue, Oct 22, 2013 at 7:49 PM, yonghu <
>>> yongyong313@gmail.com>
>>> > > > wrote:
>>> > > > > > >
>>> > > > > > > > Gray
>>> > > > > > > >
>>> > > > > > > > Thanks for your response. I tried your approach.
But it
>>> did not
>>> > > > work.
>>> > > > > > The
>>> > > > > > > > HBase just stalled, no messages, nothing happened.
By the
>>> way,
>>> > my
>>> > > > > hbase
>>> > > > > > > > version is 0.94.12.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On Tue, Oct 22, 2013 at 7:34 PM, Gary Helmling
<
>>> > > > ghelmling@gmail.com
>>> > > > > > > >wrote:
>>> > > > > > > >
>>> > > > > > > >> Within a coprocessor, you can just use
the
>>> > > CoprocessorEnvironment
>>> > > > > > > instance
>>> > > > > > > >> passed to start() method or any of the
pre/post hooks, and
>>> > call
>>> > > > > > > >> CoprocessorEnvironment.getTable(byte[]
tablename).
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Tue, Oct 22, 2013 at 9:41 AM, Ted Yu
<
>>> yuzhihong@gmail.com>
>>> > > > > wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Take a look at
>>> > > > > http://hbase.apache.org/book.html#client.connections,
>>> > > > > > > >> > especially 9.3.1.1.
>>> > > > > > > >> >
>>> > > > > > > >> >
>>> > > > > > > >> > On Tue, Oct 22, 2013 at 9:37 AM,
yonghu <
>>> > > yongyong313@gmail.com>
>>> > > > > > > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> > > Hello,
>>> > > > > > > >> > >
>>> > > > > > > >> > > In the oldest verison of HBase
, I can get the
>>> > > HTableInterface
>>> > > > > by
>>> > > > > > > >> > > HTablePool.getTable() method.
However, in the latest
>>> Hbase
>>> > > > > > > >> > version0.94.12,
>>> > > > > > > >> > > HTablePool is deprecated. So,
I tried to use
>>> > > > HConnectionManager
>>> > > > > to
>>> > > > > > > >> create
>>> > > > > > > >> > > HTableInterface, but it does
not work. Can anyone
>>> tell me
>>> > > how
>>> > > > to
>>> > > > > > > >> create
>>> > > > > > > >> > > HTableInterface in new HBase
version? By the way,
>>> there is
>>> > > no
>>> > > > > > error
>>> > > > > > > >> > message
>>> > > > > > > >> > > when I run coprocessor.
>>> > > > > > > >> > >
>>> > > > > > > >> > > regards!
>>> > > > > > > >> > >
>>> > > > > > > >> > > Yong
>>> > > > > > > >> > >
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message