hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Parallel range scanners, not getting all rows.
Date Fri, 17 Mar 2017 21:33:13 GMT
Please also notify AsyncHBase mailing list, if you haven't done so.

2017-03-17 9:01 GMT-07:00 Kristoffer Sjögren <stoffe@gmail.com>:

> Thanks Ted, that was indeed the problem.
>
> 2017-03-17 4:29 GMT+01:00 Ted Yu <yuzhihong@gmail.com>:
> > Have you considered the empty start row ?
> >
> > 2017-03-16 10:51 GMT-07:00 Kristoffer Sjögren <stoffe@gmail.com>:
> >
> >> Thanks Ted, I have posted the question to AsyncHBase.
> >>
> >> But, if I may ask. Is my reasoning correct provided I would use the
> >> same ranges for a HBase Scan object with withStartRow (inclusive) and
> >> withStopRow (exclusive) semantics?
> >>
> >> 2017-03-16 15:08 GMT+01:00 Ted Yu <yuzhihong@gmail.com>:
> >> > Since you're using AsyncHBase, please consider posting on their
> mailing
> >> > list.
> >> >
> >> > Thanks
> >> >
> >> > 2017-03-16 7:05 GMT-07:00 Kristoffer Sjögren <stoffe@gmail.com>:
> >> >
> >> >> Hi
> >> >>
> >> >> I'm trying to scan a table using start and stop key ranges based on
a
> >> >> single byte.
> >> >>
> >> >> I'm using AsyncHBase where scanners are start key inclusive and stop
> >> >> key exclusive.
> >> >>
> >> >> So for a single byte I generate 256 scanners with key ranges [1]. The
> >> >> "last" key range use start 127 and an empty end key (in order to scan
> >> >> to the end).
> >> >>
> >> >> But I don't get all the rows from these ranges. Can somebody
> enlighten
> >> >> me on why the scanners doesn't find all the rows? I compare numbers
> of
> >> >> rows using org.apache.hadoop.hbase.mapreduce.RowCounter.
> >> >>
> >> >> Cheers,
> >> >> -Kristoffer
> >> >>
> >> >> [1] 256 key ranges with start and stop key byte.
> >> >>
> >> >> KeyRange{start=[-128], stop=[-127]}
> >> >> KeyRange{start=[-127], stop=[-126]}
> >> >> KeyRange{start=[-126], stop=[-125]}
> >> >> KeyRange{start=[-125], stop=[-124]}
> >> >> KeyRange{start=[-124], stop=[-123]}
> >> >> KeyRange{start=[-123], stop=[-122]}
> >> >> KeyRange{start=[-122], stop=[-121]}
> >> >> KeyRange{start=[-121], stop=[-120]}
> >> >> KeyRange{start=[-120], stop=[-119]}
> >> >> KeyRange{start=[-119], stop=[-118]}
> >> >> KeyRange{start=[-118], stop=[-117]}
> >> >> KeyRange{start=[-117], stop=[-116]}
> >> >> KeyRange{start=[-116], stop=[-115]}
> >> >> KeyRange{start=[-115], stop=[-114]}
> >> >> KeyRange{start=[-114], stop=[-113]}
> >> >> KeyRange{start=[-113], stop=[-112]}
> >> >> KeyRange{start=[-112], stop=[-111]}
> >> >> KeyRange{start=[-111], stop=[-110]}
> >> >> KeyRange{start=[-110], stop=[-109]}
> >> >> KeyRange{start=[-109], stop=[-108]}
> >> >> KeyRange{start=[-108], stop=[-107]}
> >> >> KeyRange{start=[-107], stop=[-106]}
> >> >> KeyRange{start=[-106], stop=[-105]}
> >> >> KeyRange{start=[-105], stop=[-104]}
> >> >> KeyRange{start=[-104], stop=[-103]}
> >> >> KeyRange{start=[-103], stop=[-102]}
> >> >> KeyRange{start=[-102], stop=[-101]}
> >> >> KeyRange{start=[-101], stop=[-100]}
> >> >> KeyRange{start=[-100], stop=[-99]}
> >> >> KeyRange{start=[-99], stop=[-98]}
> >> >> KeyRange{start=[-98], stop=[-97]}
> >> >> KeyRange{start=[-97], stop=[-96]}
> >> >> KeyRange{start=[-96], stop=[-95]}
> >> >> KeyRange{start=[-95], stop=[-94]}
> >> >> KeyRange{start=[-94], stop=[-93]}
> >> >> KeyRange{start=[-93], stop=[-92]}
> >> >> KeyRange{start=[-92], stop=[-91]}
> >> >> KeyRange{start=[-91], stop=[-90]}
> >> >> KeyRange{start=[-90], stop=[-89]}
> >> >> KeyRange{start=[-89], stop=[-88]}
> >> >> KeyRange{start=[-88], stop=[-87]}
> >> >> KeyRange{start=[-87], stop=[-86]}
> >> >> KeyRange{start=[-86], stop=[-85]}
> >> >> KeyRange{start=[-85], stop=[-84]}
> >> >> KeyRange{start=[-84], stop=[-83]}
> >> >> KeyRange{start=[-83], stop=[-82]}
> >> >> KeyRange{start=[-82], stop=[-81]}
> >> >> KeyRange{start=[-81], stop=[-80]}
> >> >> KeyRange{start=[-80], stop=[-79]}
> >> >> KeyRange{start=[-79], stop=[-78]}
> >> >> KeyRange{start=[-78], stop=[-77]}
> >> >> KeyRange{start=[-77], stop=[-76]}
> >> >> KeyRange{start=[-76], stop=[-75]}
> >> >> KeyRange{start=[-75], stop=[-74]}
> >> >> KeyRange{start=[-74], stop=[-73]}
> >> >> KeyRange{start=[-73], stop=[-72]}
> >> >> KeyRange{start=[-72], stop=[-71]}
> >> >> KeyRange{start=[-71], stop=[-70]}
> >> >> KeyRange{start=[-70], stop=[-69]}
> >> >> KeyRange{start=[-69], stop=[-68]}
> >> >> KeyRange{start=[-68], stop=[-67]}
> >> >> KeyRange{start=[-67], stop=[-66]}
> >> >> KeyRange{start=[-66], stop=[-65]}
> >> >> KeyRange{start=[-65], stop=[-64]}
> >> >> KeyRange{start=[-64], stop=[-63]}
> >> >> KeyRange{start=[-63], stop=[-62]}
> >> >> KeyRange{start=[-62], stop=[-61]}
> >> >> KeyRange{start=[-61], stop=[-60]}
> >> >> KeyRange{start=[-60], stop=[-59]}
> >> >> KeyRange{start=[-59], stop=[-58]}
> >> >> KeyRange{start=[-58], stop=[-57]}
> >> >> KeyRange{start=[-57], stop=[-56]}
> >> >> KeyRange{start=[-56], stop=[-55]}
> >> >> KeyRange{start=[-55], stop=[-54]}
> >> >> KeyRange{start=[-54], stop=[-53]}
> >> >> KeyRange{start=[-53], stop=[-52]}
> >> >> KeyRange{start=[-52], stop=[-51]}
> >> >> KeyRange{start=[-51], stop=[-50]}
> >> >> KeyRange{start=[-50], stop=[-49]}
> >> >> KeyRange{start=[-49], stop=[-48]}
> >> >> KeyRange{start=[-48], stop=[-47]}
> >> >> KeyRange{start=[-47], stop=[-46]}
> >> >> KeyRange{start=[-46], stop=[-45]}
> >> >> KeyRange{start=[-45], stop=[-44]}
> >> >> KeyRange{start=[-44], stop=[-43]}
> >> >> KeyRange{start=[-43], stop=[-42]}
> >> >> KeyRange{start=[-42], stop=[-41]}
> >> >> KeyRange{start=[-41], stop=[-40]}
> >> >> KeyRange{start=[-40], stop=[-39]}
> >> >> KeyRange{start=[-39], stop=[-38]}
> >> >> KeyRange{start=[-38], stop=[-37]}
> >> >> KeyRange{start=[-37], stop=[-36]}
> >> >> KeyRange{start=[-36], stop=[-35]}
> >> >> KeyRange{start=[-35], stop=[-34]}
> >> >> KeyRange{start=[-34], stop=[-33]}
> >> >> KeyRange{start=[-33], stop=[-32]}
> >> >> KeyRange{start=[-32], stop=[-31]}
> >> >> KeyRange{start=[-31], stop=[-30]}
> >> >> KeyRange{start=[-30], stop=[-29]}
> >> >> KeyRange{start=[-29], stop=[-28]}
> >> >> KeyRange{start=[-28], stop=[-27]}
> >> >> KeyRange{start=[-27], stop=[-26]}
> >> >> KeyRange{start=[-26], stop=[-25]}
> >> >> KeyRange{start=[-25], stop=[-24]}
> >> >> KeyRange{start=[-24], stop=[-23]}
> >> >> KeyRange{start=[-23], stop=[-22]}
> >> >> KeyRange{start=[-22], stop=[-21]}
> >> >> KeyRange{start=[-21], stop=[-20]}
> >> >> KeyRange{start=[-20], stop=[-19]}
> >> >> KeyRange{start=[-19], stop=[-18]}
> >> >> KeyRange{start=[-18], stop=[-17]}
> >> >> KeyRange{start=[-17], stop=[-16]}
> >> >> KeyRange{start=[-16], stop=[-15]}
> >> >> KeyRange{start=[-15], stop=[-14]}
> >> >> KeyRange{start=[-14], stop=[-13]}
> >> >> KeyRange{start=[-13], stop=[-12]}
> >> >> KeyRange{start=[-12], stop=[-11]}
> >> >> KeyRange{start=[-11], stop=[-10]}
> >> >> KeyRange{start=[-10], stop=[-9]}
> >> >> KeyRange{start=[-9], stop=[-8]}
> >> >> KeyRange{start=[-8], stop=[-7]}
> >> >> KeyRange{start=[-7], stop=[-6]}
> >> >> KeyRange{start=[-6], stop=[-5]}
> >> >> KeyRange{start=[-5], stop=[-4]}
> >> >> KeyRange{start=[-4], stop=[-3]}
> >> >> KeyRange{start=[-3], stop=[-2]}
> >> >> KeyRange{start=[-2], stop=[-1]}
> >> >> KeyRange{start=[-1], stop=[0]}
> >> >> KeyRange{start=[0], stop=[1]}
> >> >> KeyRange{start=[1], stop=[2]}
> >> >> KeyRange{start=[2], stop=[3]}
> >> >> KeyRange{start=[3], stop=[4]}
> >> >> KeyRange{start=[4], stop=[5]}
> >> >> KeyRange{start=[5], stop=[6]}
> >> >> KeyRange{start=[6], stop=[7]}
> >> >> KeyRange{start=[7], stop=[8]}
> >> >> KeyRange{start=[8], stop=[9]}
> >> >> KeyRange{start=[9], stop=[10]}
> >> >> KeyRange{start=[10], stop=[11]}
> >> >> KeyRange{start=[11], stop=[12]}
> >> >> KeyRange{start=[12], stop=[13]}
> >> >> KeyRange{start=[13], stop=[14]}
> >> >> KeyRange{start=[14], stop=[15]}
> >> >> KeyRange{start=[15], stop=[16]}
> >> >> KeyRange{start=[16], stop=[17]}
> >> >> KeyRange{start=[17], stop=[18]}
> >> >> KeyRange{start=[18], stop=[19]}
> >> >> KeyRange{start=[19], stop=[20]}
> >> >> KeyRange{start=[20], stop=[21]}
> >> >> KeyRange{start=[21], stop=[22]}
> >> >> KeyRange{start=[22], stop=[23]}
> >> >> KeyRange{start=[23], stop=[24]}
> >> >> KeyRange{start=[24], stop=[25]}
> >> >> KeyRange{start=[25], stop=[26]}
> >> >> KeyRange{start=[26], stop=[27]}
> >> >> KeyRange{start=[27], stop=[28]}
> >> >> KeyRange{start=[28], stop=[29]}
> >> >> KeyRange{start=[29], stop=[30]}
> >> >> KeyRange{start=[30], stop=[31]}
> >> >> KeyRange{start=[31], stop=[32]}
> >> >> KeyRange{start=[32], stop=[33]}
> >> >> KeyRange{start=[33], stop=[34]}
> >> >> KeyRange{start=[34], stop=[35]}
> >> >> KeyRange{start=[35], stop=[36]}
> >> >> KeyRange{start=[36], stop=[37]}
> >> >> KeyRange{start=[37], stop=[38]}
> >> >> KeyRange{start=[38], stop=[39]}
> >> >> KeyRange{start=[39], stop=[40]}
> >> >> KeyRange{start=[40], stop=[41]}
> >> >> KeyRange{start=[41], stop=[42]}
> >> >> KeyRange{start=[42], stop=[43]}
> >> >> KeyRange{start=[43], stop=[44]}
> >> >> KeyRange{start=[44], stop=[45]}
> >> >> KeyRange{start=[45], stop=[46]}
> >> >> KeyRange{start=[46], stop=[47]}
> >> >> KeyRange{start=[47], stop=[48]}
> >> >> KeyRange{start=[48], stop=[49]}
> >> >> KeyRange{start=[49], stop=[50]}
> >> >> KeyRange{start=[50], stop=[51]}
> >> >> KeyRange{start=[51], stop=[52]}
> >> >> KeyRange{start=[52], stop=[53]}
> >> >> KeyRange{start=[53], stop=[54]}
> >> >> KeyRange{start=[54], stop=[55]}
> >> >> KeyRange{start=[55], stop=[56]}
> >> >> KeyRange{start=[56], stop=[57]}
> >> >> KeyRange{start=[57], stop=[58]}
> >> >> KeyRange{start=[58], stop=[59]}
> >> >> KeyRange{start=[59], stop=[60]}
> >> >> KeyRange{start=[60], stop=[61]}
> >> >> KeyRange{start=[61], stop=[62]}
> >> >> KeyRange{start=[62], stop=[63]}
> >> >> KeyRange{start=[63], stop=[64]}
> >> >> KeyRange{start=[64], stop=[65]}
> >> >> KeyRange{start=[65], stop=[66]}
> >> >> KeyRange{start=[66], stop=[67]}
> >> >> KeyRange{start=[67], stop=[68]}
> >> >> KeyRange{start=[68], stop=[69]}
> >> >> KeyRange{start=[69], stop=[70]}
> >> >> KeyRange{start=[70], stop=[71]}
> >> >> KeyRange{start=[71], stop=[72]}
> >> >> KeyRange{start=[72], stop=[73]}
> >> >> KeyRange{start=[73], stop=[74]}
> >> >> KeyRange{start=[74], stop=[75]}
> >> >> KeyRange{start=[75], stop=[76]}
> >> >> KeyRange{start=[76], stop=[77]}
> >> >> KeyRange{start=[77], stop=[78]}
> >> >> KeyRange{start=[78], stop=[79]}
> >> >> KeyRange{start=[79], stop=[80]}
> >> >> KeyRange{start=[80], stop=[81]}
> >> >> KeyRange{start=[81], stop=[82]}
> >> >> KeyRange{start=[82], stop=[83]}
> >> >> KeyRange{start=[83], stop=[84]}
> >> >> KeyRange{start=[84], stop=[85]}
> >> >> KeyRange{start=[85], stop=[86]}
> >> >> KeyRange{start=[86], stop=[87]}
> >> >> KeyRange{start=[87], stop=[88]}
> >> >> KeyRange{start=[88], stop=[89]}
> >> >> KeyRange{start=[89], stop=[90]}
> >> >> KeyRange{start=[90], stop=[91]}
> >> >> KeyRange{start=[91], stop=[92]}
> >> >> KeyRange{start=[92], stop=[93]}
> >> >> KeyRange{start=[93], stop=[94]}
> >> >> KeyRange{start=[94], stop=[95]}
> >> >> KeyRange{start=[95], stop=[96]}
> >> >> KeyRange{start=[96], stop=[97]}
> >> >> KeyRange{start=[97], stop=[98]}
> >> >> KeyRange{start=[98], stop=[99]}
> >> >> KeyRange{start=[99], stop=[100]}
> >> >> KeyRange{start=[100], stop=[101]}
> >> >> KeyRange{start=[101], stop=[102]}
> >> >> KeyRange{start=[102], stop=[103]}
> >> >> KeyRange{start=[103], stop=[104]}
> >> >> KeyRange{start=[104], stop=[105]}
> >> >> KeyRange{start=[105], stop=[106]}
> >> >> KeyRange{start=[106], stop=[107]}
> >> >> KeyRange{start=[107], stop=[108]}
> >> >> KeyRange{start=[108], stop=[109]}
> >> >> KeyRange{start=[109], stop=[110]}
> >> >> KeyRange{start=[110], stop=[111]}
> >> >> KeyRange{start=[111], stop=[112]}
> >> >> KeyRange{start=[112], stop=[113]}
> >> >> KeyRange{start=[113], stop=[114]}
> >> >> KeyRange{start=[114], stop=[115]}
> >> >> KeyRange{start=[115], stop=[116]}
> >> >> KeyRange{start=[116], stop=[117]}
> >> >> KeyRange{start=[117], stop=[118]}
> >> >> KeyRange{start=[118], stop=[119]}
> >> >> KeyRange{start=[119], stop=[120]}
> >> >> KeyRange{start=[120], stop=[121]}
> >> >> KeyRange{start=[121], stop=[122]}
> >> >> KeyRange{start=[122], stop=[123]}
> >> >> KeyRange{start=[123], stop=[124]}
> >> >> KeyRange{start=[124], stop=[125]}
> >> >> KeyRange{start=[125], stop=[126]}
> >> >> KeyRange{start=[126], stop=[127]}
> >> >> KeyRange{start=[127], stop=[]}
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message