hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
Date Tue, 17 Mar 2015 21:05:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366083#comment-14366083
] 

Andrew Purtell commented on HBASE-13262:
----------------------------------------

It's not a problem in 0.98 it seems.

I tested with this: https://github.com/apurtell/hbase-hwhat/tree/0.98

{noformat}
2015-03-17 14:00:52,911 WARN  [main] util.NativeCodeLoader(62): Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2015-03-17 14:00:53,103 INFO  [main] zookeeper.RecoverableZooKeeper(121): Process identifier=hconnection-0x7afc920e
connecting to ZooKeeper ensemble=localhost:2181
2015-03-17 14:00:53,167 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(320): hconnection-0x7afc920e0x0,
quorum=localhost:2181, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
path=null
2015-03-17 14:00:53,171 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(397): hconnection-0x7afc920e-0x14c2985e89a0008
connected
2015-03-17 14:00:53,566 INFO  [main] zookeeper.RecoverableZooKeeper(121): Process identifier=hconnection-0x64198527
connecting to ZooKeeper ensemble=localhost:2181
2015-03-17 14:00:53,574 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(320): hconnection-0x641985270x0,
quorum=localhost:2181, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
path=null
2015-03-17 14:00:53,578 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(397): hconnection-0x64198527-0x14c2985e89a0009
connected
2015-03-17 14:00:54,789 INFO  [main] zookeeper.RecoverableZooKeeper(121): Process identifier=catalogtracker-on-hconnection-0x64198527
connecting to ZooKeeper ensemble=localhost:2181
2015-03-17 14:00:54,790 DEBUG [main] catalog.CatalogTracker(198): Starting catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@3559b630
2015-03-17 14:00:54,797 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(320): catalogtracker-on-hconnection-0x641985270x0,
quorum=localhost:2181, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected,
path=null
2015-03-17 14:00:54,798 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(397): catalogtracker-on-hconnection-0x64198527-0x14c2985e89a000a
connected
2015-03-17 14:00:54,799 DEBUG [main] zookeeper.ZKUtil(430): catalogtracker-on-hconnection-0x64198527-0x14c2985e89a000a,
quorum=localhost:2181, baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
2015-03-17 14:00:54,821 DEBUG [main] catalog.CatalogTracker(222): Stopping catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@3559b630
2015-03-17 14:00:54,837 INFO  [main] client.HConnectionManager$HConnectionImplementation(2246):
Closing master protocol: MasterService
2015-03-17 14:00:54,837 INFO  [main] client.HConnectionManager$HConnectionImplementation(1903):
Closing zookeeper sessionid=0x14c2985e89a0009
Write buffer size: 52428800
2015-03-17 14:01:05,010 INFO  [main] hbase.HBaseTest(118): Wrote 50000 entries
2015-03-17 14:01:12,491 INFO  [main] hbase.HBaseTest(118): Wrote 100000 entries
2015-03-17 14:01:19,860 INFO  [main] hbase.HBaseTest(118): Wrote 150000 entries
2015-03-17 14:01:27,223 INFO  [main] hbase.HBaseTest(118): Wrote 200000 entries
2015-03-17 14:01:34,772 INFO  [main] hbase.HBaseTest(118): Wrote 250000 entries
2015-03-17 14:01:41,784 INFO  [main] hbase.HBaseTest(118): Wrote 300000 entries
2015-03-17 14:01:49,003 INFO  [main] hbase.HBaseTest(118): Wrote 350000 entries
2015-03-17 14:01:56,044 INFO  [main] hbase.HBaseTest(118): Wrote 400000 entries
2015-03-17 14:02:03,331 INFO  [main] hbase.HBaseTest(118): Wrote 450000 entries
2015-03-17 14:02:10,360 INFO  [main] hbase.HBaseTest(118): Wrote 500000 entries
2015-03-17 14:02:18,349 INFO  [main] hbase.HBaseTest(118): Wrote 550000 entries
2015-03-17 14:02:26,238 INFO  [main] hbase.HBaseTest(118): Wrote 600000 entries
2015-03-17 14:02:33,430 INFO  [main] hbase.HBaseTest(118): Wrote 650000 entries
2015-03-17 14:02:40,362 INFO  [main] hbase.HBaseTest(118): Wrote 700000 entries
2015-03-17 14:02:47,358 INFO  [main] hbase.HBaseTest(118): Wrote 750000 entries
2015-03-17 14:02:54,396 INFO  [main] hbase.HBaseTest(118): Wrote 800000 entries
2015-03-17 14:03:01,735 INFO  [main] hbase.HBaseTest(118): Wrote 850000 entries
2015-03-17 14:03:08,691 INFO  [main] hbase.HBaseTest(118): Wrote 900000 entries
2015-03-17 14:03:16,234 INFO  [main] hbase.HBaseTest(118): Wrote 950000 entries
2015-03-17 14:03:22,987 INFO  [main] hbase.HBaseTest(118): Wrote 1000000 entries
2015-03-17 14:03:22,987 INFO  [main] hbase.HBaseTest(133): Wrote 1000000 entries in total
2015-03-17 14:03:22,987 INFO  [main] hbase.HBaseTest(136): Closing table used for writes
2015-03-17 14:03:23,000 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '1\x00'
2015-03-17 14:03:23,667 INFO  [main] hbase.HBaseTest(156): Saw row 108997
2015-03-17 14:03:24,013 INFO  [main] hbase.HBaseTest(156): Saw row 117997
2015-03-17 14:03:24,303 INFO  [main] hbase.HBaseTest(156): Saw row 126997
2015-03-17 14:03:24,610 INFO  [main] hbase.HBaseTest(156): Saw row 135997
2015-03-17 14:03:24,911 INFO  [main] hbase.HBaseTest(156): Saw row 144997
2015-03-17 14:03:25,181 INFO  [main] hbase.HBaseTest(156): Saw row 153997
2015-03-17 14:03:25,442 INFO  [main] hbase.HBaseTest(156): Saw row 162997
2015-03-17 14:03:25,759 INFO  [main] hbase.HBaseTest(156): Saw row 171997
2015-03-17 14:03:26,053 INFO  [main] hbase.HBaseTest(156): Saw row 180997
2015-03-17 14:03:26,351 INFO  [main] hbase.HBaseTest(156): Saw row 189998
2015-03-17 14:03:26,692 INFO  [main] hbase.HBaseTest(156): Saw row 198998
2015-03-17 14:03:26,724 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '2\x00'
2015-03-17 14:03:27,026 INFO  [main] hbase.HBaseTest(156): Saw row 207997
2015-03-17 14:03:27,318 INFO  [main] hbase.HBaseTest(156): Saw row 216997
2015-03-17 14:03:27,616 INFO  [main] hbase.HBaseTest(156): Saw row 225997
2015-03-17 14:03:27,908 INFO  [main] hbase.HBaseTest(156): Saw row 234997
2015-03-17 14:03:28,203 INFO  [main] hbase.HBaseTest(156): Saw row 243997
2015-03-17 14:03:28,510 INFO  [main] hbase.HBaseTest(156): Saw row 252997
2015-03-17 14:03:28,785 INFO  [main] hbase.HBaseTest(156): Saw row 261997
2015-03-17 14:03:29,093 INFO  [main] hbase.HBaseTest(156): Saw row 270997
2015-03-17 14:03:29,362 INFO  [main] hbase.HBaseTest(156): Saw row 279998
2015-03-17 14:03:29,642 INFO  [main] hbase.HBaseTest(156): Saw row 288998
2015-03-17 14:03:29,924 INFO  [main] hbase.HBaseTest(156): Saw row 297998
2015-03-17 14:03:29,993 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '3\x00'
2015-03-17 14:03:30,270 INFO  [main] hbase.HBaseTest(156): Saw row 306997
2015-03-17 14:03:30,542 INFO  [main] hbase.HBaseTest(156): Saw row 315997
2015-03-17 14:03:30,838 INFO  [main] hbase.HBaseTest(156): Saw row 324997
2015-03-17 14:03:31,100 INFO  [main] hbase.HBaseTest(156): Saw row 333997
2015-03-17 14:03:31,365 INFO  [main] hbase.HBaseTest(156): Saw row 342997
2015-03-17 14:03:31,648 INFO  [main] hbase.HBaseTest(156): Saw row 351997
2015-03-17 14:03:31,902 INFO  [main] hbase.HBaseTest(156): Saw row 360997
2015-03-17 14:03:32,123 INFO  [main] hbase.HBaseTest(156): Saw row 369998
2015-03-17 14:03:32,345 INFO  [main] hbase.HBaseTest(156): Saw row 378998
2015-03-17 14:03:33,285 INFO  [main] hbase.HBaseTest(156): Saw row 387998
2015-03-17 14:03:33,485 INFO  [main] hbase.HBaseTest(156): Saw row 396998
2015-03-17 14:03:33,562 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '4\x00'
2015-03-17 14:03:33,750 INFO  [main] hbase.HBaseTest(156): Saw row 405997
2015-03-17 14:03:34,012 INFO  [main] hbase.HBaseTest(156): Saw row 414997
2015-03-17 14:03:34,270 INFO  [main] hbase.HBaseTest(156): Saw row 423997
2015-03-17 14:03:34,535 INFO  [main] hbase.HBaseTest(156): Saw row 432997
2015-03-17 14:03:35,658 INFO  [main] hbase.HBaseTest(156): Saw row 441997
2015-03-17 14:03:35,917 INFO  [main] hbase.HBaseTest(156): Saw row 450997
2015-03-17 14:03:36,174 INFO  [main] hbase.HBaseTest(156): Saw row 459998
2015-03-17 14:03:36,432 INFO  [main] hbase.HBaseTest(156): Saw row 468998
2015-03-17 14:03:36,674 INFO  [main] hbase.HBaseTest(156): Saw row 477998
2015-03-17 14:03:36,918 INFO  [main] hbase.HBaseTest(156): Saw row 486998
2015-03-17 14:03:37,184 INFO  [main] hbase.HBaseTest(156): Saw row 495998
2015-03-17 14:03:37,309 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '5\x00'
2015-03-17 14:03:37,470 INFO  [main] hbase.HBaseTest(156): Saw row 504997
2015-03-17 14:03:37,803 INFO  [main] hbase.HBaseTest(156): Saw row 513997
2015-03-17 14:03:38,057 INFO  [main] hbase.HBaseTest(156): Saw row 522997
2015-03-17 14:03:38,308 INFO  [main] hbase.HBaseTest(156): Saw row 531997
2015-03-17 14:03:38,571 INFO  [main] hbase.HBaseTest(156): Saw row 540997
2015-03-17 14:03:38,826 INFO  [main] hbase.HBaseTest(156): Saw row 549998
2015-03-17 14:03:39,061 INFO  [main] hbase.HBaseTest(156): Saw row 558998
2015-03-17 14:03:39,311 INFO  [main] hbase.HBaseTest(156): Saw row 567998
2015-03-17 14:03:39,560 INFO  [main] hbase.HBaseTest(156): Saw row 576998
2015-03-17 14:03:39,893 INFO  [main] hbase.HBaseTest(156): Saw row 585998
2015-03-17 14:03:40,135 INFO  [main] hbase.HBaseTest(156): Saw row 594998
2015-03-17 14:03:40,252 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '6\x00'
2015-03-17 14:03:40,367 INFO  [main] hbase.HBaseTest(156): Saw row 603997
2015-03-17 14:03:40,626 INFO  [main] hbase.HBaseTest(156): Saw row 612997
2015-03-17 14:03:40,826 INFO  [main] hbase.HBaseTest(156): Saw row 621997
2015-03-17 14:03:41,106 INFO  [main] hbase.HBaseTest(156): Saw row 630997
2015-03-17 14:03:41,346 INFO  [main] hbase.HBaseTest(156): Saw row 639998
2015-03-17 14:03:41,514 INFO  [main] hbase.HBaseTest(156): Saw row 648998
2015-03-17 14:03:41,730 INFO  [main] hbase.HBaseTest(156): Saw row 657998
2015-03-17 14:03:42,021 INFO  [main] hbase.HBaseTest(156): Saw row 666998
2015-03-17 14:03:42,273 INFO  [main] hbase.HBaseTest(156): Saw row 675998
2015-03-17 14:03:42,516 INFO  [main] hbase.HBaseTest(156): Saw row 684998
2015-03-17 14:03:42,708 INFO  [main] hbase.HBaseTest(156): Saw row 693998
2015-03-17 14:03:42,903 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '7\x00'
2015-03-17 14:03:42,979 INFO  [main] hbase.HBaseTest(156): Saw row 702997
2015-03-17 14:03:43,184 INFO  [main] hbase.HBaseTest(156): Saw row 711997
2015-03-17 14:03:43,510 INFO  [main] hbase.HBaseTest(156): Saw row 720997
2015-03-17 14:03:43,713 INFO  [main] hbase.HBaseTest(156): Saw row 729998
2015-03-17 14:03:44,057 INFO  [main] hbase.HBaseTest(156): Saw row 738998
2015-03-17 14:03:44,254 INFO  [main] hbase.HBaseTest(156): Saw row 747998
2015-03-17 14:03:44,608 INFO  [main] hbase.HBaseTest(156): Saw row 756998
2015-03-17 14:03:44,816 INFO  [main] hbase.HBaseTest(156): Saw row 765998
2015-03-17 14:03:45,054 INFO  [main] hbase.HBaseTest(156): Saw row 774998
2015-03-17 14:03:45,340 INFO  [main] hbase.HBaseTest(156): Saw row 783998
2015-03-17 14:03:45,482 INFO  [main] hbase.HBaseTest(156): Saw row 792998
2015-03-17 14:03:45,630 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '8\x00'
2015-03-17 14:03:45,680 INFO  [main] hbase.HBaseTest(156): Saw row 801997
2015-03-17 14:03:45,901 INFO  [main] hbase.HBaseTest(156): Saw row 810997
2015-03-17 14:03:46,101 INFO  [main] hbase.HBaseTest(156): Saw row 819998
2015-03-17 14:03:46,330 INFO  [main] hbase.HBaseTest(156): Saw row 828998
2015-03-17 14:03:46,647 INFO  [main] hbase.HBaseTest(156): Saw row 837998
2015-03-17 14:03:46,875 INFO  [main] hbase.HBaseTest(156): Saw row 846998
2015-03-17 14:03:47,109 INFO  [main] hbase.HBaseTest(156): Saw row 855998
2015-03-17 14:03:47,316 INFO  [main] hbase.HBaseTest(156): Saw row 864998
2015-03-17 14:03:47,534 INFO  [main] hbase.HBaseTest(156): Saw row 873998
2015-03-17 14:03:47,722 INFO  [main] hbase.HBaseTest(156): Saw row 882998
2015-03-17 14:03:47,978 INFO  [main] hbase.HBaseTest(156): Saw row 891998
2015-03-17 14:03:48,098 DEBUG [main] client.ClientScanner(277): Advancing internal scanner
to startKey at '9\x00'
2015-03-17 14:03:48,122 INFO  [main] hbase.HBaseTest(156): Saw row 900997
2015-03-17 14:03:48,454 INFO  [main] hbase.HBaseTest(156): Saw row 909998
2015-03-17 14:03:48,664 INFO  [main] hbase.HBaseTest(156): Saw row 918998
2015-03-17 14:03:48,896 INFO  [main] hbase.HBaseTest(156): Saw row 927998
2015-03-17 14:03:49,157 INFO  [main] hbase.HBaseTest(156): Saw row 936998
2015-03-17 14:03:49,391 INFO  [main] hbase.HBaseTest(156): Saw row 945998
2015-03-17 14:03:49,579 INFO  [main] hbase.HBaseTest(156): Saw row 954998
2015-03-17 14:03:49,802 INFO  [main] hbase.HBaseTest(156): Saw row 963998
2015-03-17 14:03:50,164 INFO  [main] hbase.HBaseTest(156): Saw row 972998
2015-03-17 14:03:50,347 INFO  [main] hbase.HBaseTest(156): Saw row 981998
2015-03-17 14:03:50,617 INFO  [main] hbase.HBaseTest(156): Saw row 990998
2015-03-17 14:03:50,767 INFO  [main] hbase.HBaseTest(156): Saw row 999999
2015-03-17 14:03:50,768 INFO  [main] hbase.HBaseTest(163): Last row in Result 999999
2015-03-17 14:03:50,768 INFO  [main] hbase.HBaseTest(166): Saw 1000000 rows
2015-03-17 14:03:50,768 INFO  [main] hbase.HBaseTest(167): Saw 10000000 cells
2015-03-17 14:03:51,065 INFO  [main] hbase.HBaseTest(172): Missing 0 rows: []
{noformat}


> ResultScanner doesn't return all rows in Scan
> ---------------------------------------------
>
>                 Key: HBASE-13262
>                 URL: https://issues.apache.org/jira/browse/HBASE-13262
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 2.0.0, 1.1.0
>         Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Blocker
>             Fix For: 2.0.0, 1.1.0
>
>
> Tried to write a simple Java client again 1.1.0-SNAPSHOT.
> * Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]), for a total
of 10M cells written
> * Read back the data from the table, ensure I saw 10M cells
> Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of the actual
rows. Running against 1.0.0, returns all 10M records as expected.
> [Code I was running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message