accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From z11373 <z11...@outlook.com>
Subject Re: scan command hung
Date Wed, 30 Sep 2015 17:56:16 GMT
Thanks Josh/Keith!
To answer Josh's questions, that column family only has few rows, the other
column families that work fine, one of them also only has few rows, and
another one has million of rows. Not using locality group or any other fancy
configuration.

I tried 'debug on' per suggestion from Keith, and perhaps it's obvious to
know the culprit from the output.
Below are the snippets (the first one stopped after returning the result,
the second one kept going indefinitely):

/// on column family that works fine ///
///
2015-09-30 09:55:35,281 [shell.Shell] DEBUG: Found no scan iterators to set
2015-09-30 09:55:35,307 [impl.TabletLocatorImpl] TRACE: tid=12 oid=7 
Locating tablet  table=22 row=  skipRow=false retry=false
2015-09-30 09:55:35,308 [impl.TabletLocatorImpl] TRACE: tid=12 oid=8 
Locating tablet  table=!0 row=22;  skipRow=false retry=false
2015-09-30 09:55:35,308 [impl.RootTabletLocator] TRACE: tid=12 oid=9 
Looking up root tablet location in zookeeper.
2015-09-30 09:55:35,310 [impl.RootTabletLocator] TRACE: tid=12 oid=9  Found
root tablet at server01:10014|14eb7844d280183 in 0.001 secs
2015-09-30 09:55:35,315 [metadata.MetadataLocationObtainer] TRACE: tid=12
oid=10  Looking up in +r row=!0;22;  extent=+r<< tserver=server01:10014
2015-09-30 09:55:35,320 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,359 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 562
2015-09-30 09:55:35,360 [metadata.MetadataLocationObtainer] TRACE: tid=12
oid=10  Got 4 results  from +r<< in 0.045 secs
2015-09-30 09:55:35,362 [impl.TabletLocatorImpl] TRACE: tid=12 oid=8 
Located tablet !0;~< at server01:10014 in 0.054 secs
2015-09-30 09:55:35,363 [metadata.MetadataLocationObtainer] TRACE: tid=12
oid=11  Looking up in !0 row=22;  extent=!0;~< tserver=server01:10014
2015-09-30 09:55:35,363 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,369 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 720
2015-09-30 09:55:35,369 [metadata.MetadataLocationObtainer] TRACE: tid=12
oid=11  Got 8 results  from !0;~< in 0.006 secs
2015-09-30 09:55:35,370 [impl.TabletLocatorImpl] TRACE: tid=12 oid=7 
Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at
server01:10014 in 0.063 secs
2015-09-30 09:55:35,370 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,371 [impl.ThriftScanner] TRACE: tid=12 oid=12  Starting
scan tserver=server01:10014
tablet=22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< range=(-inf,+inf)
ssil=[] ssio={}
2015-09-30 09:55:35,387 [impl.ThriftScanner] TRACE: tid=12 oid=12  Finished
scanning tablet in 0.016 secs #results=0
2015-09-30 09:55:35,387 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 272
2015-09-30 09:55:35,387 [impl.TabletLocatorImpl] TRACE: tid=12 oid=13 
Locating tablet  table=22
row=sehttp://dbpedia.org/resource/2008%E2%80%9309_M  skipRow=true
retry=false
2015-09-30 09:55:35,387 [impl.TabletLocatorImpl] TRACE: tid=12 oid=13 
Located tablet
22;sehttp://dbpedia.org/resource/James_Co;sehttp://dbpedia.org/resource/2008%E2%80%9309_M
at server01:10014 in 0.000 secs
2015-09-30 09:55:35,388 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,388 [impl.ThriftScanner] TRACE: tid=12 oid=14  Starting
scan tserver=server01:10014
tablet=22;sehttp://dbpedia.org/resource/James_Co;sehttp://dbpedia.org/resource/2008%E2%80%9309_M
range=(-inf,+inf) ssil=[] ssio={}
2015-09-30 09:55:35,392 [impl.ThriftScanner] TRACE: tid=12 oid=14  Finished
scanning tablet in 0.004 secs #results=0
2015-09-30 09:55:35,392 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 278
2015-09-30 09:55:35,392 [impl.TabletLocatorImpl] TRACE: tid=12 oid=15 
Locating tablet  table=22 row=sehttp://dbpedia.org/resource/James_Co 
skipRow=true retry=false
2015-09-30 09:55:35,393 [impl.TabletLocatorImpl] TRACE: tid=12 oid=15 
Located tablet
22;sehttp://dbpedia.org/resource/Royal_L;sehttp://dbpedia.org/resource/James_Co
at server01:10014 in 0.001 secs
2015-09-30 09:55:35,393 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,393 [impl.ThriftScanner] TRACE: tid=12 oid=16  Starting
scan tserver=server01:10014
tablet=22;sehttp://dbpedia.org/resource/Royal_L;sehttp://dbpedia.org/resource/James_Co
range=(-inf,+inf) ssil=[] ssio={}
2015-09-30 09:55:35,396 [impl.ThriftScanner] TRACE: tid=12 oid=16  Finished
scanning tablet in 0.003 secs #results=0
2015-09-30 09:55:35,396 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 278
2015-09-30 09:55:35,396 [impl.TabletLocatorImpl] TRACE: tid=12 oid=17 
Locating tablet  table=22 row=sehttp://dbpedia.org/resource/Royal_L 
skipRow=true retry=false
2015-09-30 09:55:35,396 [impl.TabletLocatorImpl] TRACE: tid=12 oid=17 
Located tablet 22<;sehttp://dbpedia.org/resource/Royal_L at server01:10014
in 0.000 secs
2015-09-30 09:55:35,397 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,397 [impl.ThriftScanner] TRACE: tid=12 oid=18  Starting
scan tserver=server01:10014 tablet=22<;sehttp://dbpedia.org/resource/Royal_L
range=(-inf,+inf) ssil=[] ssio={}
2015-09-30 09:55:35,410 [impl.ThriftScanner] TRACE: tid=12 oid=18 
Completely finished scan in 0.013 secs #results=0
2015-09-30 09:55:35,410 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 272
2015-09-30 09:55:35,410 [admin.TableOperations] TRACE: tid=12 oid=19 
Fetching list of tables...
2015-09-30 09:55:35,412 [admin.TableOperations] TRACE: tid=12 oid=19 
Fetched 23 table names in 0.002 secs
2015-09-30 09:55:35,413 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:55:35,413 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 138
2015-09-30 09:55:35,413 [admin.TableOperations] TRACE: tid=12 oid=20 
Fetching list of namespaces...
2015-09-30 09:55:35,414 [admin.TableOperations] TRACE: tid=12 oid=20 
Fetched 2 namespaces in 0.001 secs



/// on column family that hung ///
///
2015-09-30 09:59:33,609 [admin.TableOperations] TRACE: tid=12 oid=25 
Checking if table BVAULT_RDF_STATS exists...
2015-09-30 09:59:33,612 [admin.TableOperations] TRACE: tid=12 oid=25 
Checked existance of true in 0.002 secs
2015-09-30 09:59:33,613 [impl.ThriftTransportPool] TRACE: Creating new
connection to connection to server01:10014
2015-09-30 09:59:33,628 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 4968
2015-09-30 09:59:33,628 [admin.TableOperations] TRACE: tid=12 oid=26 
Checking if table BVAULT_RDF_STATS exists...
2015-09-30 09:59:33,630 [admin.TableOperations] TRACE: tid=12 oid=26 
Checked existance of true in 0.002 secs
2015-09-30 09:59:33,630 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:33,644 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 4968
2015-09-30 09:59:33,644 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:33,658 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 4968
2015-09-30 09:59:33,658 [admin.TableOperations] TRACE: tid=12 oid=27 
Checking if table BVAULT_RDF_STATS exists...
2015-09-30 09:59:33,660 [admin.TableOperations] TRACE: tid=12 oid=27 
Checked existance of true in 0.002 secs
2015-09-30 09:59:33,661 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:33,675 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 4968
2015-09-30 09:59:33,675 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:33,676 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 130
2015-09-30 09:59:33,678 [shell.Shell] DEBUG: Found no scan iterators to set
2015-09-30 09:59:33,682 [impl.TabletLocatorImpl] TRACE: tid=12 oid=28 
Locating tablet  table=22 row=  skipRow=false retry=false
2015-09-30 09:59:33,682 [impl.TabletLocatorImpl] TRACE: tid=12 oid=28 
Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at
server01:10014 in 0.000 secs
2015-09-30 09:59:33,682 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:33,682 [impl.ThriftScanner] TRACE: tid=12 oid=29  Starting
scan tserver=server01:10014
tablet=22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< range=(-inf,+inf)
ssil=[] ssio={}
2015-09-30 09:59:34,685 [impl.ThriftScanner] TRACE: tid=12 oid=29  Finished
scan in 1.003 secs #results=0 scanid=4197848475135257903
2015-09-30 09:59:34,685 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 242
2015-09-30 09:59:34,685 [impl.TabletLocatorImpl] TRACE: tid=12 oid=30 
Locating tablet  table=22 row=  skipRow=false retry=false
2015-09-30 09:59:34,686 [impl.TabletLocatorImpl] TRACE: tid=12 oid=30 
Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at
server01:10014 in 0.000 secs
2015-09-30 09:59:34,686 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:34,686 [impl.ThriftScanner] TRACE: tid=12 oid=31 
Continuing scan tserver=server01:10014 scanid=4197848475135257903
2015-09-30 09:59:35,696 [impl.ThriftScanner] TRACE: tid=12 oid=31  Finished
scan in 1.010 secs #results=0 scanid=4197848475135257903
2015-09-30 09:59:35,696 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 122
2015-09-30 09:59:35,696 [impl.TabletLocatorImpl] TRACE: tid=12 oid=32 
Locating tablet  table=22 row=  skipRow=false retry=false
2015-09-30 09:59:35,697 [impl.TabletLocatorImpl] TRACE: tid=12 oid=32 
Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at
server01:10014 in 0.001 secs
2015-09-30 09:59:35,697 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:35,697 [impl.ThriftScanner] TRACE: tid=12 oid=33 
Continuing scan tserver=server01:10014 scanid=4197848475135257903
2015-09-30 09:59:36,699 [impl.ThriftScanner] TRACE: tid=12 oid=33  Finished
scan in 1.001 secs #results=0 scanid=4197848475135257903
2015-09-30 09:59:36,699 [impl.ThriftTransportPool] TRACE: Returned
connection server01:10014 (120000) ioCount : 122
2015-09-30 09:59:36,699 [impl.TabletLocatorImpl] TRACE: tid=12 oid=34 
Locating tablet  table=22 row=  skipRow=false retry=false
2015-09-30 09:59:36,699 [impl.TabletLocatorImpl] TRACE: tid=12 oid=34 
Located tablet 22;sehttp://dbpedia.org/resource/2008%E2%80%9309_M< at
server01:10014 in 0.000 secs
2015-09-30 09:59:36,699 [impl.ThriftTransportPool] TRACE: Using existing
connection to server01:10014
2015-09-30 09:59:36,699 [impl.ThriftScanner] TRACE: tid=12 oid=35 
Continuing scan tserver=server01:10014 scanid=4197848475135257903
2015-09-30 09:59:37,701 [impl.ThriftScanner] TRACE: tid=12 oid=35  Finished
scan in 1.002 secs #results=0 scanid=4197848475135257903
...
...
...
(keep going)



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/scan-command-hung-tp15286p15296.html
Sent from the Developers mailing list archive at Nabble.com.

Mime
View raw message