hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop John <anoop.hb...@gmail.com>
Subject Re: Error about rs block seek
Date Mon, 13 May 2013 07:47:04 GMT
> Current pos = 32651;
currKeyLen = 45; currValLen = 80; block limit = 32775

This means after the cur position we need to have atleast  45+80+4(key
length stored as 4 bytes) +4(value length 4 bytes)
So atleast 32784 should have been the limit.  If we have memstoreTS also
written with this KV some more bytes..

Do u use Hbase handled checksum?

-Anoop-

On Mon, May 13, 2013 at 12:00 PM, Bing Jiang <jiangbinglover@gmail.com>wrote:

> Hi,all
> Before the exception stack, there is an Error log:
> 2013-05-13 00:00:14,491 ERROR
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2: Current pos = 32651;
> currKeyLen = 45; currValLen = 80; block limit = 32775; HFile name =
> 1f96183d55144c058fa2a05fe5c0b814; currBlock currBlockOffset = 33550830
>
> And the operation is scanner's next.
> Current pos + currKeyLen + currValLen > block limit
> 32651+45 +80 = 32776 > 32775 , and in my table configs, set blocksize
> 32768, and when I change the value from blocksize from 64k(default value)
> to 32k, so many error logs being found.
>
> I use 0.94.3, can someone tell me the influence of blocksize setting.
>
> Tks.
>
>
>
>
> 2013/5/13 ramkrishna vasudevan <ramkrishna.s.vasudevan@gmail.com>
>
> > Your TTL is negative here 'TTL => '-1','.
> >
> > Any reason for it to be negative? This could be a possible reason.  Not
> > sure..
> >
> > Regards
> > Ram
> >
> >
> > On Mon, May 13, 2013 at 7:20 AM, Bing Jiang <jiangbinglover@gmail.com
> > >wrote:
> >
> > > hi, Ted.
> > >
> > > No data block encoding, our table config below:
> > >
> > > User Table Description
> > > CrawlInfo<http://10.100.12.33:8003/table.jsp?name=CrawlInfo> {NAME
> > > => 'CrawlInfo', DEFERRED_LOG_FLUSH => 'true', MAX_FILESIZE =>
> > > '34359738368', FAMILIES => [{NAME => 'CrawlStats', BLOOMFILTER =>
> > 'ROWCOL',
> > > CACHE_INDEX_ON_WRITE => 'true', TTL => '-1', CACHE_DATA_ON_WRITE =>
> > 'true',
> > > CACHE_BLOOMS_ON_WRITE => 'true', VERSIONS => '1', BLOCKSIZE =>
> '32768'}]}
> > >
> > >
> > >
> > > 2013/5/13 Bing Jiang <jiangbinglover@gmail.com>
> > >
> > > > Hi, JM.
> > > > Our jdk version is 1.6.0_38
> > > >
> > > >
> > > > 2013/5/13 Jean-Marc Spaggiari <jean-marc@spaggiari.org>
> > > >
> > > >> Hi Bing,
> > > >>
> > > >> Which JDK are you using?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> JM
> > > >>
> > > >> 2013/5/12 Bing Jiang <jiangbinglover@gmail.com>
> > > >>
> > > >> > Yes, we use hbase-0.94.3 , and  we change block.size from 64k
to
> > 32k.
> > > >> >
> > > >> >
> > > >> > 2013/5/13 Ted Yu <yuzhihong@gmail.com>
> > > >> >
> > > >> > > Can you tell us the version of hbase you are using ?
> > > >> > > Did this problem happen recently ?
> > > >> > >
> > > >> > > Thanks
> > > >> > >
> > > >> > > On May 12, 2013, at 6:25 PM, Bing Jiang <
> jiangbinglover@gmail.com
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Hi, all.
> > > >> > > > In our hbase cluster, there are many logs like below:
> > > >> > > >
> > > >> > > > 2013-05-13 00:00:04,161 ERROR
> > > >> > > org.apache.hadoop.hbase.regionserver.HRegionServer:
> > > >> > > > java.lang.IllegalArgumentException
> > > >> > > >         at java.nio.Buffer.position(Buffer.java:216)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.blockSeek(HFileReaderV2.java:882)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:753)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:487)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:131)
> > > >> > > >         at
> > > >> > >
> > > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2073)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3412)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1642)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1634)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1610)
> > > >> > > >         at
> > > >> > >
> > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4230)
> > > >> > > >         at
> > > >> > >
> > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4204)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2025)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3461)
> > > >> > > >         at
> sun.reflect.GeneratedMethodAccessor30.invoke(Unknown
> > > >> Source)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > >> > > >         at java.lang.reflect.Method.invoke(Method.java:597)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> > > >> > > >         at
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > and Table config:
> > > >> > > >
> > > >> > > >
> > > >> > > > Can anyone tell me how I can find the reason about
this?
> > > >> > > >
> > > >> > > > --
> > > >> > > > Bing Jiang
> > > >> > > > weibo: http://weibo.com/jiangbinglover
> > > >> > > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > >> > > > National Research Center for Intelligent Computing
Systems
> > > >> > > > Institute of Computing technology
> > > >> > > > Graduate University of Chinese Academy of Science
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Bing Jiang
> > > >> > Tel:(86)134-2619-1361
> > > >> > weibo: http://weibo.com/jiangbinglover
> > > >> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > >> > National Research Center for Intelligent Computing Systems
> > > >> > Institute of Computing technology
> > > >> > Graduate University of Chinese Academy of Science
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Bing Jiang
> > > > Tel:(86)134-2619-1361
> > > > weibo: http://weibo.com/jiangbinglover
> > > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > > National Research Center for Intelligent Computing Systems
> > > > Institute of Computing technology
> > > > Graduate University of Chinese Academy of Science
> > > >
> > >
> > >
> > >
> > > --
> > > Bing Jiang
> > > Tel:(86)134-2619-1361
> > > weibo: http://weibo.com/jiangbinglover
> > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > National Research Center for Intelligent Computing Systems
> > > Institute of Computing technology
> > > Graduate University of Chinese Academy of Science
> > >
> >
>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: http://blog.sina.com.cn/jiangbinglover
> National Research Center for Intelligent Computing Systems
> Institute of Computing technology
> Graduate University of Chinese Academy of Science
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message