Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of anoop.hbase@gmail.com
 designates 209.85.128.43 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62zSs5Xeoom3=n5+Fw5d+K=qHOXPHeEsToo8Ze-UuxggDA@mail.gmail.com>
References: 
 <CAKnDEhiF4GgKJf0sPf6uJS+QcF=fBGxW57EswFDL+rE+oCVqFw@mail.gmail.com>
	<CALte62zSs5Xeoom3=n5+Fw5d+K=qHOXPHeEsToo8Ze-UuxggDA@mail.gmail.com>
Date: Sat, 2 Mar 2013 23:14:46 +0530
Message-ID: 
 <CAOtJ30pUuGDr39bKpT4qdOLA3RV+Fpi7+g-7S95Dnc4L2ETR1g@mail.gmail.com>
Subject: Re: Concurrently Reading Still Got Exceptions
From: Anoop John <anoop.hbase@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=00504502c133fd3dc604d6f4addc

--00504502c133fd3dc604d6f4addc
Content-Type: text/plain; charset=ISO-8859-1

Is this really related to concurrent reads?  I think some thing else..
Will dig into code tomorrow.  Can you attach a junit test case which will
produce NPE.

-Anoop-


On Sat, Mar 2, 2013 at 9:29 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looks like the issue might be related to HTable:
>
>           at org.apache.hadoop.hbase.client.HTable$ClientScanner.
> nextScanner(HTable.java:1167)
>           at org.apache.hadoop.hbase.client.HTable$ClientScanner.
> next(HTable.java:1296)
>           at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.
> hasNext(HTable.java:1356)
>
> In newer version of HBase (0.94), you can pass executor to HTable ctor so
> that you don't need to use HTablePool:
>
>   public HTable(Configuration conf, final byte[] tableName,
> finalExecutorService pool)
>
> Cheers
>
> On Wed, Feb 6, 2013 at 2:27 AM, Bing Li <lblabs@gmail.com> wrote:
>
> > Dear all,
> >
> > Some exceptions are raised when I concurrently read data from HBase.
> > The version of HBase I used is 0.92.0.
> >
> > I cannot fix the problem. Could you please help me?
> >
> > Thanks so much!
> >
> > Best wishes,
> > Bing
> >
> >       Feb 6, 2013 12:21:31 AM
> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection run
> >       WARNING: Unexpected exception receiving call responses
> > java.lang.NullPointerException
> >           at
> >
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
> >           at
> >
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> >       Feb 6, 2013 12:21:31 AM
> > org.apache.hadoop.hbase.client.ScannerCallable close
> >       WARNING: Ignore, probably already closed
> >       java.io.IOException: Call to greatfreeweb/127.0.1.1:60020
> > failed on local exception: java.io.IOException: Unexpected exception
> > receiving call responses
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:934)
> >           at
> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:903)
> >           at
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> >           at $Proxy6.close(Unknown Source)
> >           at
> >
> org.apache.hadoop.hbase.client.ScannerCallable.close(ScannerCallable.java:112)
> >           at
> >
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:74)
> >           at
> >
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:39)
> >           at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1325)
> >           at
> >
> org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1167)
> >           at
> >
> org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1296)
> >           at
> >
> org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1356)
> >           at
> >
> com.greatfree.hbase.rank.NodeRankRetriever.loadNodeGroupNodeRankRowKeys(NodeRankRetriever.java:348)
> >           at
> >
> com.greatfree.ranking.PersistNodeGroupNodeRanksThread.run(PersistNodeGroupNodeRanksThread.java:29)
> >           at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >           at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >           at java.lang.Thread.run(Thread.java:662) Caused by:
> > java.io.IOException: Unexpected exception receiving call responses
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:509)
> >       Caused by: java.lang.NullPointerException
> >           at
> >
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
> >           at
> >
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
> >           at
> >
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> >
> > I read data from HBase concurrently with the following code.
> >
> >         ...
> >                 ExecutorService threadPool =
> > Executors.newFixedThreadPool(100);
> >                 LoadNodeGroupNodeRankRowKeyThread thread;
> >                 Set<String> groupKeys;
> >                 for (String nodeKey : nodeKeys)
> >                 {
> >                         groupKeys =
> NodeCache.WWW().getGroupKeys(nodeKey);
> >                         for (String groupKey : groupKeys)
> >                         {
> >                                 // Threads are initialized and executed
> > here.
> >                                 thread = new
> > LoadNodeGroupNodeRankRowKeyThread(nodeKey, groupKey,
> > TimingScale.PERMANENTLY);
> >                                 threadPool.execute(thread);
> >                         }
> >                 }
> >                 Scanner in = new Scanner(System.in);
> >                 in.nextLine();
> >                 threadPool.shutdownNow();
> >         ...
> >
> > The code of LoadNodeGroupNodeRankRowKeyThread is as follows,
> >
> >         ...
> >         public void run()
> >         {
> >                 NodeRankRetriever retriever = new NodeRankRetriever();
> >                 Set<String> rowKeys =
> > retriever.loadNodeGroupNodeRankRowKeys(this.hostNodeKey,
> > this.groupKey, this.timingScale);
> >                 if (rowKeys.size() > 0)
> >                 {
> >                         for (String rowKey : rowKeys)
> >                         {
> >                                 System.out.println(rowKey);
> >                         }
> >                 }
> >                 else
> >                 {
> >                         System.out.println("No data loaded");
> >                 }
> >                 retriever.dispose();
> >         }
> >         ...
> >
> > The constructor of NodeRankRetriever() just got an instance of HTable
> > from HTablePool from the following method.
> >
> >         ...
> >         public HTableInterface getTable(String tableName)
> >         {
> >                 return this.hTablePool.getTable(tableName);
> >         }
> >         ...
> >
> > The method dispose() of NodeRankRetriever() just close the
> > HTableInterface created by HTablePool.
> >
> >         ...
> >         public void dispose()
> >         {
> >                 try
> >                 {
> >                         this.rankTable.close();
> >                 }
> >                 catch (IOException e)
> >                 {
> >                         e.printStackTrace();
> >                 }
> >         }
> >         ...
> >
>

--00504502c133fd3dc604d6f4addc--