hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: long garbage collecting pause
Date Tue, 02 Oct 2012 13:38:55 GMT

El 01/10/2012 16:35, Greg Ross escribió:
> Hi,
>
> I'm having difficulty with a mapreduce job that has reducers that read
> from and write to HBase, version 0.92.1, r1298924. Row sizes vary
> greatly. As do the number of cells, although the number of cells is
> typically numbered in the tens, at most. The max cell size is 1MB.
0.94.1 is out with a lot of improvements related to performance. It 
would better
if you use tis version.
>
> I see the following in the logs followed by the region server promptly
> shutting down:
>
> 2012-10-01 19:08:47,858 [regionserver60020] WARN
> org.apache.hadoop.hbase.util.Sleeper: We slept 28970ms instead of
> 3000ms, this is likely due to a long garbage collecting pause and it's
> usually bad, see
> http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
>
> The full logs, including GC are below.
>
> Although new to HBase, I've read up on the likely GC issues and their
> remedies. I've implemented the recommended solutions and still to no
> avail.
>
> Here's what I've tried:
>
> (1) increased the RAM to 4G
Which is the exact size of your RAM?
> (2) set -XX:+UseConcMarkSweepGC
> (3) set -XX:+UseParNewGC
> (4) set -XX:CMSInitiatingOccupancyFraction=N where I've attempted N=[40..70]
> (5) I've called context.progress() in the reducer before and after
> reading and writing
> (6) memstore is enabled
>
> Is there anything else that I might have missed?
>
> Thanks,
>
> Greg
I´m seeing in the HBase logs that there a lot of block requests that are 
failing.
Can you send to us the ouput of JPS?
Can you check the filesystem´s health with hadoop fsck?

Another thing that I´m seeing is that one of your main process is 
compaction,
so you can optimize all this inceasing the size of your regions (by 
defaulf the size of a
region is 256 MB), but you will have in your hands a "split/compaction 
storm" like
Lars called them on his book.

Instead using the default mechanism for region spliting and compaction, 
you can turn it
off and do it manually with the split and major_compaction commands.

You can evaluate to use compresion in your cluster to save a lot of 
space in your region servers.

Which is the size of your cluster?
You can use SPM, Ganglia or OpenTSDB to monitor constantly your cluster.

Best wishes
>
> hbase logs
> ========
>
> 2012-10-01 19:09:48,293
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/.tmp/d2ee47650b224189b0c27d1c20929c03
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> 2012-10-01 19:09:48,884
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 5 file(s) in U of
> orwell_events,00321084,1349118541283.a9906c96a91bb8d7e62a7a528bf0ea5c.
> into d2ee47650b224189b0c27d1c20929c03, size=723.0m; total size for
> store is 723.0m
> 2012-10-01 19:09:48,884
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,00321084,1349118541283.a9906c96a91bb8d7e62a7a528bf0ea5c.,
> storeName=U, fileCount=5, fileSize=1.4g, priority=2,
> time=10631266687564968; duration=35sec
> 2012-10-01 19:09:48,886
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.
> 2012-10-01 19:09:48,887
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 5
> file(s) in U of
> orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/.tmp,
> seqid=132201184, totalSize=1.4g
> 2012-10-01 19:10:04,191
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/.tmp/2e5534fea8b24eaf9cc1e05dea788c01
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> 2012-10-01 19:10:04,868
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 5 file(s) in U of
> orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.
> into 2e5534fea8b24eaf9cc1e05dea788c01, size=626.5m; total size for
> store is 626.5m
> 2012-10-01 19:10:04,868
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.,
> storeName=U, fileCount=5, fileSize=1.4g, priority=2,
> time=10631266696614208; duration=15sec
> 2012-10-01 19:14:04,992
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56469014,1348940247961.6ace1f2f0b7ad3e454f738d66255047f.
> 2012-10-01 19:14:04,993
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56469014,1348940247961.6ace1f2f0b7ad3e454f738d66255047f.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/6ace1f2f0b7ad3e454f738d66255047f/.tmp,
> seqid=132198830, totalSize=863.8m
> 2012-10-01 19:14:19,147
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/6ace1f2f0b7ad3e454f738d66255047f/.tmp/b741f8501ad248418c48262d751f6e86
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/6ace1f2f0b7ad3e454f738d66255047f/U/b741f8501ad248418c48262d751f6e86
> 2012-10-01 19:14:19,381
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56469014,1348940247961.6ace1f2f0b7ad3e454f738d66255047f.
> into b741f8501ad248418c48262d751f6e86, size=851.4m; total size for
> store is 851.4m
> 2012-10-01 19:14:19,381
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56469014,1348940247961.6ace1f2f0b7ad3e454f738d66255047f.,
> storeName=U, fileCount=2, fileSize=863.8m, priority=5,
> time=10631557965747111; duration=14sec
> 2012-10-01 19:14:19,381
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56329744,1349028887914.9686d8348fe53644334c0423cc217d26.
> 2012-10-01 19:14:19,381
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56329744,1349028887914.9686d8348fe53644334c0423cc217d26.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9686d8348fe53644334c0423cc217d26/.tmp,
> seqid=132198819, totalSize=496.7m
> 2012-10-01 19:14:27,337
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9686d8348fe53644334c0423cc217d26/.tmp/78040c736c4149a884a1bdcda9916416
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9686d8348fe53644334c0423cc217d26/U/78040c736c4149a884a1bdcda9916416
> 2012-10-01 19:14:27,514
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56329744,1349028887914.9686d8348fe53644334c0423cc217d26.
> into 78040c736c4149a884a1bdcda9916416, size=487.5m; total size for
> store is 487.5m
> 2012-10-01 19:14:27,514
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56329744,1349028887914.9686d8348fe53644334c0423cc217d26.,
> storeName=U, fileCount=3, fileSize=496.7m, priority=4,
> time=10631557966599560; duration=8sec
> 2012-10-01 19:14:27,514
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56601774,1349029123008.34b52e7208034f85db8d1e39ca6c1329.
> 2012-10-01 19:14:27,514
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56601774,1349029123008.34b52e7208034f85db8d1e39ca6c1329.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/34b52e7208034f85db8d1e39ca6c1329/.tmp,
> seqid=132200816, totalSize=521.7m
> 2012-10-01 19:14:36,962
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/34b52e7208034f85db8d1e39ca6c1329/.tmp/0142b8bcdda948c185887358990af6d1
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/34b52e7208034f85db8d1e39ca6c1329/U/0142b8bcdda948c185887358990af6d1
> 2012-10-01 19:14:37,171
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56601774,1349029123008.34b52e7208034f85db8d1e39ca6c1329.
> into 0142b8bcdda948c185887358990af6d1, size=510.7m; total size for
> store is 510.7m
> 2012-10-01 19:14:37,171
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56601774,1349029123008.34b52e7208034f85db8d1e39ca6c1329.,
> storeName=U, fileCount=3, fileSize=521.7m, priority=4,
> time=10631557967125617; duration=9sec
> 2012-10-01 19:14:37,172
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56586234,1349028973106.44449344385d98cd7512008dfa532f8e.
> 2012-10-01 19:14:37,172
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56586234,1349028973106.44449344385d98cd7512008dfa532f8e.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/44449344385d98cd7512008dfa532f8e/.tmp,
> seqid=132198832, totalSize=565.5m
> 2012-10-01 19:14:57,082
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/44449344385d98cd7512008dfa532f8e/.tmp/44a27dce8df04306908579c22be76786
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/44449344385d98cd7512008dfa532f8e/U/44a27dce8df04306908579c22be76786
> 2012-10-01 19:14:57,429
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56586234,1349028973106.44449344385d98cd7512008dfa532f8e.
> into 44a27dce8df04306908579c22be76786, size=557.7m; total size for
> store is 557.7m
> 2012-10-01 19:14:57,429
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56586234,1349028973106.44449344385d98cd7512008dfa532f8e.,
> storeName=U, fileCount=3, fileSize=565.5m, priority=4,
> time=10631557967207683; duration=20sec
> 2012-10-01 19:14:57,429
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56342344,1348941231573.8adf268d4fcb494344745c14b090e773.
> 2012-10-01 19:14:57,430
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56342344,1348941231573.8adf268d4fcb494344745c14b090e773.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8adf268d4fcb494344745c14b090e773/.tmp,
> seqid=132199414, totalSize=845.6m
> 2012-10-01 19:16:54,394
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8adf268d4fcb494344745c14b090e773/.tmp/771813ba0c87449ebd99d5e7916244f8
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8adf268d4fcb494344745c14b090e773/U/771813ba0c87449ebd99d5e7916244f8
> 2012-10-01 19:16:54,636
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56342344,1348941231573.8adf268d4fcb494344745c14b090e773.
> into 771813ba0c87449ebd99d5e7916244f8, size=827.3m; total size for
> store is 827.3m
> 2012-10-01 19:16:54,636
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56342344,1348941231573.8adf268d4fcb494344745c14b090e773.,
> storeName=U, fileCount=3, fileSize=845.6m, priority=4,
> time=10631557967560440; duration=1mins, 57sec
> 2012-10-01 19:16:54,636
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56538694,1348941049708.b9f56cf1f6f6c7b0cdf2a07a3d36846b.
> 2012-10-01 19:16:54,637
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56538694,1348941049708.b9f56cf1f6f6c7b0cdf2a07a3d36846b.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/b9f56cf1f6f6c7b0cdf2a07a3d36846b/.tmp,
> seqid=132198824, totalSize=1012.4m
> 2012-10-01 19:17:35,610
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/b9f56cf1f6f6c7b0cdf2a07a3d36846b/.tmp/771a4124c763468c8dea927cb53887ee
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/b9f56cf1f6f6c7b0cdf2a07a3d36846b/U/771a4124c763468c8dea927cb53887ee
> 2012-10-01 19:17:35,874
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56538694,1348941049708.b9f56cf1f6f6c7b0cdf2a07a3d36846b.
> into 771a4124c763468c8dea927cb53887ee, size=974.0m; total size for
> store is 974.0m
> 2012-10-01 19:17:35,875
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56538694,1348941049708.b9f56cf1f6f6c7b0cdf2a07a3d36846b.,
> storeName=U, fileCount=3, fileSize=1012.4m, priority=4,
> time=10631557967678796; duration=41sec
> 2012-10-01 19:17:35,875
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56332944,1349028887914.506f6865d3167d722fec947a59761822.
> 2012-10-01 19:17:35,875
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56332944,1349028887914.506f6865d3167d722fec947a59761822.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/506f6865d3167d722fec947a59761822/.tmp,
> seqid=132198815, totalSize=530.5m
> 2012-10-01 19:17:47,481
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/506f6865d3167d722fec947a59761822/.tmp/24328f8244f747bf8ba81b74ef2893fa
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/506f6865d3167d722fec947a59761822/U/24328f8244f747bf8ba81b74ef2893fa
> 2012-10-01 19:17:47,741
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56332944,1349028887914.506f6865d3167d722fec947a59761822.
> into 24328f8244f747bf8ba81b74ef2893fa, size=524.0m; total size for
> store is 524.0m
> 2012-10-01 19:17:47,741
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56332944,1349028887914.506f6865d3167d722fec947a59761822.,
> storeName=U, fileCount=3, fileSize=530.5m, priority=4,
> time=10631557967807915; duration=11sec
> 2012-10-01 19:17:47,741
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56597943,1349029123008.bf64051a387fc2970252a1c8919dfd88.
> 2012-10-01 19:17:47,741
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56597943,1349029123008.bf64051a387fc2970252a1c8919dfd88.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/bf64051a387fc2970252a1c8919dfd88/.tmp,
> seqid=132201190, totalSize=529.3m
> 2012-10-01 19:17:58,031
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/bf64051a387fc2970252a1c8919dfd88/.tmp/cae48d1b96eb4440a7bcd5fa3b4c070b
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/bf64051a387fc2970252a1c8919dfd88/U/cae48d1b96eb4440a7bcd5fa3b4c070b
> 2012-10-01 19:17:58,232
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56597943,1349029123008.bf64051a387fc2970252a1c8919dfd88.
> into cae48d1b96eb4440a7bcd5fa3b4c070b, size=521.3m; total size for
> store is 521.3m
> 2012-10-01 19:17:58,232
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56597943,1349029123008.bf64051a387fc2970252a1c8919dfd88.,
> storeName=U, fileCount=3, fileSize=529.3m, priority=4,
> time=10631557967959079; duration=10sec
> 2012-10-01 19:17:58,232
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56717254,1349029064698.31a1dcefef4d5e3133b323cdaac918d7.
> 2012-10-01 19:17:58,232
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 3
> file(s) in U of
> orwell_events,56717254,1349029064698.31a1dcefef4d5e3133b323cdaac918d7.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31a1dcefef4d5e3133b323cdaac918d7/.tmp,
> seqid=132199205, totalSize=475.2m
> 2012-10-01 19:18:06,764
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31a1dcefef4d5e3133b323cdaac918d7/.tmp/ba51afdc860048b6b2e1047b06fb3b29
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31a1dcefef4d5e3133b323cdaac918d7/U/ba51afdc860048b6b2e1047b06fb3b29
> 2012-10-01 19:18:07,065
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 3 file(s) in U of
> orwell_events,56717254,1349029064698.31a1dcefef4d5e3133b323cdaac918d7.
> into ba51afdc860048b6b2e1047b06fb3b29, size=474.5m; total size for
> store is 474.5m
> 2012-10-01 19:18:07,065
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56717254,1349029064698.31a1dcefef4d5e3133b323cdaac918d7.,
> storeName=U, fileCount=3, fileSize=475.2m, priority=4,
> time=10631557968104570; duration=8sec
> 2012-10-01 19:18:07,065
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56447305,1349027937173.fb85da01ca228de9f9ac6ffa63416e9b.
> 2012-10-01 19:18:07,065
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56447305,1349027937173.fb85da01ca228de9f9ac6ffa63416e9b.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/fb85da01ca228de9f9ac6ffa63416e9b/.tmp,
> seqid=132198822, totalSize=522.5m
> 2012-10-01 19:18:18,306
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/fb85da01ca228de9f9ac6ffa63416e9b/.tmp/7a0bd16b11f34887b2690e9510071bf0
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/fb85da01ca228de9f9ac6ffa63416e9b/U/7a0bd16b11f34887b2690e9510071bf0
> 2012-10-01 19:18:18,439
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56447305,1349027937173.fb85da01ca228de9f9ac6ffa63416e9b.
> into 7a0bd16b11f34887b2690e9510071bf0, size=520.0m; total size for
> store is 520.0m
> 2012-10-01 19:18:18,440
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56447305,1349027937173.fb85da01ca228de9f9ac6ffa63416e9b.,
> storeName=U, fileCount=2, fileSize=522.5m, priority=5,
> time=10631557965863914; duration=11sec
> 2012-10-01 19:18:18,440
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56511684,1349027965795.f0198c0a2a34f18da689910235a9b0e2.
> 2012-10-01 19:18:18,440
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56511684,1349027965795.f0198c0a2a34f18da689910235a9b0e2.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f0198c0a2a34f18da689910235a9b0e2/.tmp,
> seqid=132198823, totalSize=548.0m
> 2012-10-01 19:18:32,288
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f0198c0a2a34f18da689910235a9b0e2/.tmp/dcd050acc2e747738a90aebaae8920e4
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f0198c0a2a34f18da689910235a9b0e2/U/dcd050acc2e747738a90aebaae8920e4
> 2012-10-01 19:18:32,431
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56511684,1349027965795.f0198c0a2a34f18da689910235a9b0e2.
> into dcd050acc2e747738a90aebaae8920e4, size=528.2m; total size for
> store is 528.2m
> 2012-10-01 19:18:32,431
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56511684,1349027965795.f0198c0a2a34f18da689910235a9b0e2.,
> storeName=U, fileCount=2, fileSize=548.0m, priority=5,
> time=10631557966071838; duration=13sec
> 2012-10-01 19:18:32,431
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56427793,1349031095620.88fa75b4719f4b83b9165474139c4a94.
> 2012-10-01 19:18:32,431
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56427793,1349031095620.88fa75b4719f4b83b9165474139c4a94.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/88fa75b4719f4b83b9165474139c4a94/.tmp,
> seqid=132199001, totalSize=475.9m
> 2012-10-01 19:18:43,154
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/88fa75b4719f4b83b9165474139c4a94/.tmp/15a9167cd9754fd4b3674fe732648a03
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/88fa75b4719f4b83b9165474139c4a94/U/15a9167cd9754fd4b3674fe732648a03
> 2012-10-01 19:18:43,322
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56427793,1349031095620.88fa75b4719f4b83b9165474139c4a94.
> into 15a9167cd9754fd4b3674fe732648a03, size=475.9m; total size for
> store is 475.9m
> 2012-10-01 19:18:43,322
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56427793,1349031095620.88fa75b4719f4b83b9165474139c4a94.,
> storeName=U, fileCount=2, fileSize=475.9m, priority=5,
> time=10631557966273447; duration=10sec
> 2012-10-01 19:18:43,322
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56388374,1348941251489.8c5f4903b82a4ff64ff1638c95692b60.
> 2012-10-01 19:18:43,322
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56388374,1348941251489.8c5f4903b82a4ff64ff1638c95692b60.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8c5f4903b82a4ff64ff1638c95692b60/.tmp,
> seqid=132198833, totalSize=824.8m
> 2012-10-01 19:19:00,252
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8c5f4903b82a4ff64ff1638c95692b60/.tmp/bf8da91da0824a909f684c3ecd0ee8da
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/8c5f4903b82a4ff64ff1638c95692b60/U/bf8da91da0824a909f684c3ecd0ee8da
> 2012-10-01 19:19:00,788
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56388374,1348941251489.8c5f4903b82a4ff64ff1638c95692b60.
> into bf8da91da0824a909f684c3ecd0ee8da, size=803.0m; total size for
> store is 803.0m
> 2012-10-01 19:19:00,788
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56388374,1348941251489.8c5f4903b82a4ff64ff1638c95692b60.,
> storeName=U, fileCount=2, fileSize=824.8m, priority=5,
> time=10631557966382580; duration=17sec
> 2012-10-01 19:19:00,788
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56414764,1349027912141.74a852e5b4186edd51ca714bd77f80c0.
> 2012-10-01 19:19:00,788
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56414764,1349027912141.74a852e5b4186edd51ca714bd77f80c0.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/74a852e5b4186edd51ca714bd77f80c0/.tmp,
> seqid=132198810, totalSize=565.3m
> 2012-10-01 19:19:11,311
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/74a852e5b4186edd51ca714bd77f80c0/.tmp/5cd2032f48bc4287b8866165dcb6d3e6
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/74a852e5b4186edd51ca714bd77f80c0/U/5cd2032f48bc4287b8866165dcb6d3e6
> 2012-10-01 19:19:11,504
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56414764,1349027912141.74a852e5b4186edd51ca714bd77f80c0.
> into 5cd2032f48bc4287b8866165dcb6d3e6, size=553.5m; total size for
> store is 553.5m
> 2012-10-01 19:19:11,504
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56414764,1349027912141.74a852e5b4186edd51ca714bd77f80c0.,
> storeName=U, fileCount=2, fileSize=565.3m, priority=5,
> time=10631557966480961; duration=10sec
> 2012-10-01 19:19:11,504
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,5649502,1349027928183.2088f23f8fb1dbc67b972f8744aca289.
> 2012-10-01 19:19:11,504
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,5649502,1349027928183.2088f23f8fb1dbc67b972f8744aca289.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/2088f23f8fb1dbc67b972f8744aca289/.tmp,
> seqid=132198825, totalSize=519.6m
> 2012-10-01 19:19:22,186
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/2088f23f8fb1dbc67b972f8744aca289/.tmp/6f29b3b15f1747c196ac9aa79f4835b1
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/2088f23f8fb1dbc67b972f8744aca289/U/6f29b3b15f1747c196ac9aa79f4835b1
> 2012-10-01 19:19:22,437
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,5649502,1349027928183.2088f23f8fb1dbc67b972f8744aca289.
> into 6f29b3b15f1747c196ac9aa79f4835b1, size=512.7m; total size for
> store is 512.7m
> 2012-10-01 19:19:22,437
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,5649502,1349027928183.2088f23f8fb1dbc67b972f8744aca289.,
> storeName=U, fileCount=2, fileSize=519.6m, priority=5,
> time=10631557966769107; duration=10sec
> 2012-10-01 19:19:22,437
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56487344,1349027988535.cd7e7eb88967b3dcb223de9c4ad807a9.
> 2012-10-01 19:19:22,437
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56487344,1349027988535.cd7e7eb88967b3dcb223de9c4ad807a9.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/cd7e7eb88967b3dcb223de9c4ad807a9/.tmp,
> seqid=132198836, totalSize=528.3m
> 2012-10-01 19:19:34,752
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/cd7e7eb88967b3dcb223de9c4ad807a9/.tmp/d836630f7e2b4212848d7e4edc7238f1
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/cd7e7eb88967b3dcb223de9c4ad807a9/U/d836630f7e2b4212848d7e4edc7238f1
> 2012-10-01 19:19:34,945
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56487344,1349027988535.cd7e7eb88967b3dcb223de9c4ad807a9.
> into d836630f7e2b4212848d7e4edc7238f1, size=504.3m; total size for
> store is 504.3m
> 2012-10-01 19:19:34,945
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56487344,1349027988535.cd7e7eb88967b3dcb223de9c4ad807a9.,
> storeName=U, fileCount=2, fileSize=528.3m, priority=5,
> time=10631557967026388; duration=12sec
> 2012-10-01 19:19:34,945
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56394773,1348941251489.f81d0498ab42b400f37a48d4f3854006.
> 2012-10-01 19:19:34,945
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56394773,1348941251489.f81d0498ab42b400f37a48d4f3854006.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f81d0498ab42b400f37a48d4f3854006/.tmp,
> seqid=132198841, totalSize=813.8m
> 2012-10-01 19:19:49,303
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f81d0498ab42b400f37a48d4f3854006/.tmp/c70692c971cd4e899957f9d5b189372e
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/f81d0498ab42b400f37a48d4f3854006/U/c70692c971cd4e899957f9d5b189372e
> 2012-10-01 19:19:49,428
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56394773,1348941251489.f81d0498ab42b400f37a48d4f3854006.
> into c70692c971cd4e899957f9d5b189372e, size=813.7m; total size for
> store is 813.7m
> 2012-10-01 19:19:49,428
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56394773,1348941251489.f81d0498ab42b400f37a48d4f3854006.,
> storeName=U, fileCount=2, fileSize=813.8m, priority=5,
> time=10631557967436197; duration=14sec
> 2012-10-01 19:19:49,428
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56532464,1348941298909.31c8b60bb6ad6840de937a28e3482101.
> 2012-10-01 19:19:49,429
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56532464,1348941298909.31c8b60bb6ad6840de937a28e3482101.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31c8b60bb6ad6840de937a28e3482101/.tmp,
> seqid=132198642, totalSize=812.0m
> 2012-10-01 19:20:38,718
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31c8b60bb6ad6840de937a28e3482101/.tmp/bf99f97891ed42f7847a11cfb8f46438
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/31c8b60bb6ad6840de937a28e3482101/U/bf99f97891ed42f7847a11cfb8f46438
> 2012-10-01 19:20:38,825
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56532464,1348941298909.31c8b60bb6ad6840de937a28e3482101.
> into bf99f97891ed42f7847a11cfb8f46438, size=811.3m; total size for
> store is 811.3m
> 2012-10-01 19:20:38,825
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56532464,1348941298909.31c8b60bb6ad6840de937a28e3482101.,
> storeName=U, fileCount=2, fileSize=812.0m, priority=5,
> time=10631557968183922; duration=49sec
> 2012-10-01 19:20:38,826
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,95507615,1349009451986.c08d16db6188bd8cec100eeb1291d5b9.
> 2012-10-01 19:20:38,826
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,95507615,1349009451986.c08d16db6188bd8cec100eeb1291d5b9.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/c08d16db6188bd8cec100eeb1291d5b9/.tmp,
> seqid=132198138, totalSize=588.7m
> 2012-10-01 19:20:48,274
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/c08d16db6188bd8cec100eeb1291d5b9/.tmp/9f44b7eeab58407ca98bb4ec90126035
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/c08d16db6188bd8cec100eeb1291d5b9/U/9f44b7eeab58407ca98bb4ec90126035
> 2012-10-01 19:20:48,383
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,95507615,1349009451986.c08d16db6188bd8cec100eeb1291d5b9.
> into 9f44b7eeab58407ca98bb4ec90126035, size=573.4m; total size for
> store is 573.4m
> 2012-10-01 19:20:48,383
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,95507615,1349009451986.c08d16db6188bd8cec100eeb1291d5b9.,
> storeName=U, fileCount=2, fileSize=588.7m, priority=5,
> time=10631557968302831; duration=9sec
> 2012-10-01 19:20:48,383
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56355104,1348941138879.326405233b2b444691860b14ef587f78.
> 2012-10-01 19:20:48,383
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56355104,1348941138879.326405233b2b444691860b14ef587f78.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/326405233b2b444691860b14ef587f78/.tmp,
> seqid=132198644, totalSize=870.8m
> 2012-10-01 19:21:04,998
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/326405233b2b444691860b14ef587f78/.tmp/920844c25b1847c6ac4b880e8cf1d5b0
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/326405233b2b444691860b14ef587f78/U/920844c25b1847c6ac4b880e8cf1d5b0
> 2012-10-01 19:21:05,107
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56355104,1348941138879.326405233b2b444691860b14ef587f78.
> into 920844c25b1847c6ac4b880e8cf1d5b0, size=869.0m; total size for
> store is 869.0m
> 2012-10-01 19:21:05,107
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56355104,1348941138879.326405233b2b444691860b14ef587f78.,
> storeName=U, fileCount=2, fileSize=870.8m, priority=5,
> time=10631557968521590; duration=16sec
> 2012-10-01 19:21:05,107
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56728994,1348946404223.99012cf45da4109e6b570e8b0178852c.
> 2012-10-01 19:21:05,107
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56728994,1348946404223.99012cf45da4109e6b570e8b0178852c.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/99012cf45da4109e6b570e8b0178852c/.tmp,
> seqid=132198622, totalSize=885.3m
> 2012-10-01 19:21:27,231
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/99012cf45da4109e6b570e8b0178852c/.tmp/c85d413975d642fc914253bd08f3484f
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/99012cf45da4109e6b570e8b0178852c/U/c85d413975d642fc914253bd08f3484f
> 2012-10-01 19:21:27,791
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56728994,1348946404223.99012cf45da4109e6b570e8b0178852c.
> into c85d413975d642fc914253bd08f3484f, size=848.3m; total size for
> store is 848.3m
> 2012-10-01 19:21:27,791
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56728994,1348946404223.99012cf45da4109e6b570e8b0178852c.,
> storeName=U, fileCount=2, fileSize=885.3m, priority=5,
> time=10631557968628383; duration=22sec
> 2012-10-01 19:21:27,791
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on U
> in region orwell_events,56819704,1348962077056.621ebefbdb194a82d6314ff0f58b67b1.
> 2012-10-01 19:21:27,791
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 2
> file(s) in U of
> orwell_events,56819704,1348962077056.621ebefbdb194a82d6314ff0f58b67b1.
> into tmpdir=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/621ebefbdb194a82d6314ff0f58b67b1/.tmp,
> seqid=132198621, totalSize=796.5m
> 2012-10-01 19:21:42,374
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Renaming compacted file at
> hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/621ebefbdb194a82d6314ff0f58b67b1/.tmp/ce543c630dd142309af6dca2a9ab5786
> to hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/621ebefbdb194a82d6314ff0f58b67b1/U/ce543c630dd142309af6dca2a9ab5786
> 2012-10-01 19:21:42,515
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction
> of 2 file(s) in U of
> orwell_events,56819704,1348962077056.621ebefbdb194a82d6314ff0f58b67b1.
> into ce543c630dd142309af6dca2a9ab5786, size=795.5m; total size for
> store is 795.5m
> 2012-10-01 19:21:42,516
> [regionserver60020-largeCompactions-1348577979539] INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed compaction:
> regionName=orwell_events,56819704,1348962077056.621ebefbdb194a82d6314ff0f58b67b1.,
> storeName=U, fileCount=2, fileSize=796.5m, priority=5,
> time=10631557968713853; duration=14sec
> 2012-10-01 19:49:58,159 [ResponseProcessor for block
> blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: DFSOutputStream ResponseProcessor
> exception  for block
> blk_5535637699691880681_51616301java.io.EOFException
>      at java.io.DataInputStream.readFully(DataInputStream.java:180)
>      at java.io.DataInputStream.readLong(DataInputStream.java:399)
>      at org.apache.hadoop.hdfs.protocol.DataTransferProtocol$PipelineAck.readFields(DataTransferProtocol.java:120)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2634)
>
> 2012-10-01 19:49:58,167 [IPC Server handler 87 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: (operationTooSlow):
> {"processingtimems":46208,"client":"10.100.102.155:38534","timeRange":[0,9223372036854775807],"starttimems":1349120951956,"responsesize":329939,"class":"HRegionServer","table":"orwell_events","cacheBlocks":true,"families":{"U":["ALL"]},"row":"00322994","queuetimems":0,"method":"get","totalColumns":1,"maxVersions":1}
> 2012-10-01 19:49:58,160
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have
> not heard from server in 56633ms for sessionid 0x137ec64368509f7,
> closing socket connection and attempting reconnect
> 2012-10-01 19:49:58,160 [regionserver60020] WARN
> org.apache.hadoop.hbase.util.Sleeper: We slept 49116ms instead of
> 3000ms, this is likely due to a long garbage collecting pause and it's
> usually bad, see
> http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
> 2012-10-01 19:49:58,160
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Client session timed out, have
> not heard from server in 53359ms for sessionid 0x137ec64368509f6,
> closing socket connection and attempting reconnect
> 2012-10-01 19:49:58,320 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] INFO
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 waiting for responder to exit.
> 2012-10-01 19:49:58,380 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:49:58,380 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:49:59,113 [regionserver60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: Unhandled
> exception: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT
> rejected; currently processing
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272 as dead server
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272 as dead server
>      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>      at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>      at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>      at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:797)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:688)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272 as dead server
>      at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:222)
>      at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:148)
>      at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:844)
>      at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:918)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      at $Proxy8.regionServerReport(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:794)
>      ... 2 more
> 2012-10-01 19:49:59,114 [regionserver60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:49:59,397 [IPC Server handler 36 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: (operationTooSlow):
> {"processingtimems":47521,"client":"10.100.102.176:60221","timeRange":[0,9223372036854775807],"starttimems":1349120951875,"responsesize":699312,"class":"HRegionServer","table":"orwell_events","cacheBlocks":true,"families":{"U":["ALL"]},"row":"00318223","queuetimems":0,"method":"get","totalColumns":1,"maxVersions":1}
> 2012-10-01 19:50:00,355 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #0 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:00,355
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to
> server namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181
> 2012-10-01 19:50:00,356
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-10-01 19:50:00,356 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 1 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010. Will
> retry...
> 2012-10-01 19:50:00,357
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181, initiating
> session
> 2012-10-01 19:50:00,358
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old
> server; r-o mode will be unavailable
> 2012-10-01 19:50:00,359 [regionserver60020-EventThread] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272:
> regionserver:60020-0x137ec64368509f6-0x137ec64368509f6
> regionserver:60020-0x137ec64368509f6-0x137ec64368509f6 received
> expired from ZooKeeper, aborting
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired
>      at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:374)
>      at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:271)
>      at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
>      at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2012-10-01 19:50:00,359
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper
> service, session 0x137ec64368509f6 has expired, closing socket
> connection
> 2012-10-01 19:50:00,359 [regionserver60020-EventThread] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:00,367 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1201,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:00,367 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1201,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:00,381
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to
> server namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181
> 2012-10-01 19:50:00,401 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled
> exception: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT
> rejected; currently processing
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272 as dead server
> 2012-10-01 19:50:00,403
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-10-01 19:50:00,412 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> regionserver:60020-0x137ec64368509f6-0x137ec64368509f6
> regionserver:60020-0x137ec64368509f6-0x137ec64368509f6 received
> expired from ZooKeeper, aborting
> 2012-10-01 19:50:00,412 [regionserver60020-EventThread] INFO
> org.apache.zookeeper.ClientCnxn: EventThread shut down
> 2012-10-01 19:50:00,412 [regionserver60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
> 2012-10-01 19:50:00,413
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181, initiating
> session
> 2012-10-01 19:50:00,413 [IPC Server handler 9 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 20 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 20 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 2 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 10 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 10 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server listener on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on
> 60020
> 2012-10-01 19:50:00,413 [IPC Server handler 12 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 12 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 21 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 21 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 13 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 13 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 19 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 19 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 22 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 22 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 11 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 11 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: Sending interrupt
> to stop the worker thread
> 2012-10-01 19:50:00,414 [IPC Server handler 6 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping
> infoServer
> 2012-10-01 19:50:00,414 [IPC Server handler 0 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 28 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 28 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 7 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 15 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 15 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 5 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 48 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 48 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 14 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 14 on 60020:
> exiting
> 2012-10-01 19:50:00,413 [IPC Server handler 18 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 18 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 37 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 37 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 47 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 47 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 50 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 50 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 45 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 45 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 36 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 36 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 43 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 43 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 42 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 42 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 38 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 38 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 8 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 40 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 40 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 34 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 34 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 4 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60020:
> exiting
> 2012-10-01 19:50:00,415 [IPC Server handler 44 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@5fa9b60a,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00320394"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.117:56438: output error
> 2012-10-01 19:50:00,414 [IPC Server handler 61 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59104
> remote=/10.100.101.156:50010]. 59988 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:176)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:193)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1243)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,415 [IPC Server handler 44 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 44 on 60020
> caught: java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:00,415 [IPC Server handler 44 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 44 on 60020:
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 31 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 31 on 60020:
> exiting
> 2012-10-01 19:50:00,414
> [SplitLogWorker-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272]
> INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker:
> SplitLogWorker interrupted while waiting for task, exiting:
> java.lang.InterruptedException
> 2012-10-01 19:50:00,563
> [SplitLogWorker-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272]
> INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker:
> SplitLogWorker data3024.ngpipes.milp.ngmoco.com,60020,1341428844272
> exiting
> 2012-10-01 19:50:00,414 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 3201413024070455305:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59115
> remote=/10.100.101.156:50010]. 59999 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readShort(DataInputStream.java:295)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.newBlockReader(DFSClient.java:1478)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2041)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,563 [IPC Server Responder] INFO
> org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
> 2012-10-01 19:50:00,414 [IPC Server handler 27 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 27 on 60020:
> exiting
> 2012-10-01 19:50:00,414
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old
> server; r-o mode will be unavailable
> 2012-10-01 19:50:00,414 [IPC Server handler 55 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block -2144655386884254555:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59108
> remote=/10.100.101.156:50010]. 60000 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1350)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,649
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper
> service, session 0x137ec64368509f7 has expired, closing socket
> connection
> 2012-10-01 19:50:00,414 [IPC Server handler 39 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.173:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/f6f6314e944144b7a752222f83f33ede
> for block -2100467641393578191:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:48825
> remote=/10.100.102.173:50010]. 60000 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,414 [IPC Server handler 26 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -5183799322211896791:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59078
> remote=/10.100.101.156:50010]. 59949 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,414 [IPC Server handler 85 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -5183799322211896791:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59082
> remote=/10.100.101.156:50010]. 59950 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,414 [IPC Server handler 57 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -1763662403960466408:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59116
> remote=/10.100.101.156:50010]. 60000 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readShort(DataInputStream.java:295)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.newBlockReader(DFSClient.java:1478)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2041)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,649 [IPC Server handler 79 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 2209451090614340242:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,649 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> This client just lost it's session with ZooKeeper, trying to
> reconnect.
> 2012-10-01 19:50:00,649 [IPC Server handler 89 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,649 [PRI IPC Server handler 3 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 3 on 60020:
> exiting
> 2012-10-01 19:50:00,649 [PRI IPC Server handler 0 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 0 on 60020:
> exiting
> 2012-10-01 19:50:00,700 [IPC Server handler 56 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 56 on 60020:
> exiting
> 2012-10-01 19:50:00,649 [PRI IPC Server handler 2 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 2 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 54 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 54 on 60020:
> exiting
> 2012-10-01 19:50:00,563 [IPC Server Responder] INFO
> org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
> 2012-10-01 19:50:00,701 [IPC Server handler 71 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 71 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 79 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.193:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 2209451090614340242:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,563 [IPC Server handler 16 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 4946845190538507957:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,701 [PRI IPC Server handler 9 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 9 on 60020:
> exiting
> 2012-10-01 19:50:00,563 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,415 [IPC Server handler 60 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@7eee7b96,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00321525"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.125:49043: output error
> 2012-10-01 19:50:00,704 [IPC Server handler 3 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 6550563574061266649:java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,717 [IPC Server handler 49 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 49 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 94 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 94 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 83 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 83 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [PRI IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 1 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [PRI IPC Server handler 7 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 7 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 82 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 82 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [PRI IPC Server handler 6 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 6 on 60020:
> exiting
> 2012-10-01 19:50:00,719 [IPC Server handler 16 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.107:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 4946845190538507957:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,701 [IPC Server handler 74 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 74 on 60020:
> exiting
> 2012-10-01 19:50:00,719 [IPC Server handler 86 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 86 on 60020:
> exiting
> 2012-10-01 19:50:00,721 [IPC Server handler 60 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 60 on 60020
> caught: java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:00,721 [IPC Server handler 60 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 60 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [PRI IPC Server handler 5 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 5 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [regionserver60020] INFO org.mortbay.log:
> Stopped SelectChannelConnector@0.0.0.0:60030
> 2012-10-01 19:50:00,722 [IPC Server handler 35 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 35 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 16 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.133:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 4946845190538507957:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,722 [IPC Server handler 98 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 98 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 68 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 68 on 60020:
> exiting
> 2012-10-01 19:50:00,701 [IPC Server handler 64 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 64 on 60020:
> exiting
> 2012-10-01 19:50:00,673 [IPC Server handler 33 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 33 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 76 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 76 on 60020:
> exiting
> 2012-10-01 19:50:00,673 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> Trying to reconnect to zookeeper
> 2012-10-01 19:50:00,736 [IPC Server handler 84 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 84 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 95 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 95 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 75 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 75 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 92 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 92 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 88 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 88 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 67 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 67 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 30 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 30 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 80 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 80 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 62 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 62 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 52 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 52 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 32 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 32 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 97 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 97 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 96 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 96 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 93 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 93 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 73 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 73 on 60020:
> exiting
> 2012-10-01 19:50:00,722 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.47:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,722 [IPC Server handler 87 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 87 on 60020:
> exiting
> 2012-10-01 19:50:00,721 [IPC Server handler 81 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 81 on 60020:
> exiting
> 2012-10-01 19:50:00,721 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,721 [IPC Server handler 59 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block -9081461281107361903:java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 65 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 65 on 60020:
> exiting
> 2012-10-01 19:50:00,721 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedChannelException
>      at java.nio.channels.spi.AbstractSelectableChannel.configureBlocking(AbstractSelectableChannel.java:252)
>      at org.apache.hadoop.net.SocketIOWithTimeout.<init>(SocketIOWithTimeout.java:66)
>      at org.apache.hadoop.net.SocketInputStream$Reader.<init>(SocketInputStream.java:50)
>      at org.apache.hadoop.net.SocketInputStream.<init>(SocketInputStream.java:73)
>      at org.apache.hadoop.net.SocketInputStream.<init>(SocketInputStream.java:91)
>      at org.apache.hadoop.net.NetUtils.getInputStream(NetUtils.java:323)
>      at org.apache.hadoop.net.NetUtils.getInputStream(NetUtils.java:299)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.newBlockReader(DFSClient.java:1474)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2041)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,721 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59074
> remote=/10.100.101.156:50010]. 59947 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,811 [IPC Server handler 59 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.135:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block -9081461281107361903:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59107
> remote=/10.100.101.156:50010]. 60000 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,831 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.153:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 39 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.144:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/f6f6314e944144b7a752222f83f33ede
> for block -2100467641393578191:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 26 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.138:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -5183799322211896791:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,852 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.174:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block 5946486101046455013:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59091
> remote=/10.100.101.156:50010]. 59953 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)

>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:185)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
>      at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2865)
>      at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1434)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1426)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1402)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3692)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.148:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 3201413024070455305:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 53 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 53 on 60020:
> exiting
> 2012-10-01 19:50:00,719 [IPC Server handler 79 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.154:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 2209451090614340242:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 89 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.47:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,719 [IPC Server handler 46 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 4946845190538507957:java.io.InterruptedIOException:
> Interruped while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/10.100.101.156:59113
> remote=/10.100.101.156:50010]. 59999 millis timeout left.
>      at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readShort(DataInputStream.java:295)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.newBlockReader(DFSClient.java:1478)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2041)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,895 [IPC Server handler 26 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.139:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -5183799322211896791:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,701 [IPC Server handler 91 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 91 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 3 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.114:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 6550563574061266649:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,899 [IPC Server handler 89 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.134:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,717 [PRI IPC Server handler 4 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 4 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 77 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 77 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [PRI IPC Server handler 8 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 8 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 99 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 99 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 85 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.138:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -5183799322211896791:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,717 [IPC Server handler 51 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 51 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 57 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.138:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -1763662403960466408:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,717 [IPC Server handler 55 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.180:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block -2144655386884254555:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,717 [IPC Server handler 70 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 70 on 60020:
> exiting
> 2012-10-01 19:50:00,717 [IPC Server handler 61 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.174:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 55 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.173:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block -2144655386884254555:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,705 [IPC Server handler 23 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -1763662403960466408:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,705 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,704 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:270)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
>      at java.io.DataInputStream.read(DataInputStream.java:132)
>      at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:141)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1389)
>      at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
>      at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
>      at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.read(DFSClient.java:1249)
>      at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:384)
>      at org.apache.hadoop.hdfs.DFSClient$BlockReader.readAll(DFSClient.java:1522)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2047)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 61 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.97:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 23 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.144:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -1763662403960466408:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [regionserver60020-EventThread] INFO
> org.apache.zookeeper.ZooKeeper: Initiating client connection,
> connectString=namenode-sn301.ngpipes.milp.ngmoco.com:2181
> sessionTimeout=180000 watcher=hconnection
> 2012-10-01 19:50:00,905 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.72:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 55 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-2144655386884254555_51616216 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,904 [IPC Server handler 57 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.144:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block -1763662403960466408:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,901 [IPC Server handler 85 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-5183799322211896791_51616591 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,899 [IPC Server handler 89 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_5937357897784147544_51616546 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,899 [IPC Server handler 3 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_6550563574061266649_51616152 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,896 [IPC Server handler 46 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_4946845190538507957_51616628 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,896 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.133:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block 5946486101046455013:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:185)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
>      at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2865)
>      at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1434)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1426)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1402)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3692)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,896 [IPC Server handler 26 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-5183799322211896791_51616591 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,896 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.175:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 3201413024070455305:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,895 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.97:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,894 [IPC Server handler 39 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.151:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/f6f6314e944144b7a752222f83f33ede
> for block -2100467641393578191:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,894 [IPC Server handler 79 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_2209451090614340242_51616188 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,857 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.101:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,856 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.134:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,839 [IPC Server handler 59 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.194:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block -9081461281107361903:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,811 [IPC Server handler 16 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_4946845190538507957_51616628 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,787 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.134:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,780 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.134:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,736 [IPC Server handler 63 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 63 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 72 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 72 on 60020:
> exiting
> 2012-10-01 19:50:00,736 [IPC Server handler 78 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 78 on 60020:
> exiting
> 2012-10-01 19:50:00,906 [IPC Server handler 59 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-9081461281107361903_51616031 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,906 [IPC Server handler 39 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-2100467641393578191_51531005 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,906 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.145:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block 5946486101046455013:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:185)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
>      at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2865)
>      at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1434)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1426)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1402)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3692)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,905 [IPC Server handler 57 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-1763662403960466408_51616605 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,905 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.162:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 23 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_-1763662403960466408_51616605 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,904 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.72:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:00,904 [IPC Server handler 61 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_1768076108943205533_51616106 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:00,941 [regionserver60020-SendThread()] INFO
> org.apache.zookeeper.ClientCnxn: Opening socket connection to server
> /10.100.102.197:2181
> 2012-10-01 19:50:00,941 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier
> of this process is 20776@data3024.ngpipes.milp.ngmoco.com
> 2012-10-01 19:50:00,942
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-10-01 19:50:00,943
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181, initiating
> session
> 2012-10-01 19:50:00,962
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old
> server; r-o mode will be unavailable
> 2012-10-01 19:50:00,962
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
> on server namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181,
> sessionid = 0x137ec64373dd4b3, negotiated timeout = 40000
> 2012-10-01 19:50:00,971 [regionserver60020-EventThread] INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> Reconnected successfully. This disconnect could have been caused by a
> network partition or a long-running GC pause, either way it's
> recommended that you verify your environment.
> 2012-10-01 19:50:00,971 [regionserver60020-EventThread] INFO
> org.apache.zookeeper.ClientCnxn: EventThread shut down
> 2012-10-01 19:50:01,018 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block 5946486101046455013:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:185)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
>      at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2865)
>      at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1434)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1426)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1402)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3692)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,018 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,019 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.133:50010 for file
> /hbase/orwell_events/6463149a16179d4e44c19bb49e4b4a81/U/021d9dde273e4e60ac3f8a1411a206be
> for block 5946486101046455013:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:185)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:83)
>      at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1721)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:2865)
>      at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1434)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1426)
>      at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1402)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3692)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,019 [IPC Server handler 41 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_5946486101046455013_51616031 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:01,020 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.162:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 2851854722247682142:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,021 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,023 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.47:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,023 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.47:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,024 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.174:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,024 [IPC Server handler 61 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@20c6e4bc,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00321393"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.118:57165: output error
> 2012-10-01 19:50:01,038 [IPC Server handler 61 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 61 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,038 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.134:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 5937357897784147544:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,038 [IPC Server handler 61 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 61 on 60020:
> exiting
> 2012-10-01 19:50:01,038 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.148:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 3201413024070455305:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,039 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.97:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 1768076108943205533:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,039 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.153:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,039 [IPC Server handler 66 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_1768076108943205533_51616106 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:01,039 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.102.101:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,041 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.156:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,042 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.153:50010 for file
> /hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
> for block 8387547514055202675:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,044 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed to connect to
> /10.100.101.175:50010 for file
> /hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
> for block 3201413024070455305:java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2035)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> 2012-10-01 19:50:01,090 [IPC Server handler 29 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not seek StoreFileScanner[HFileScanner for
> reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00321084/U:BAHAMUTIOS_1/1348883706322/Put,
> lastKey=00324324/U:user/1348900694793/Put, avgKeyLen=31,
> avgValueLen=125185, entries=6053, length=758129544,
> cur=00321312/U:KINGDOMSQUESTSIPAD_2/1349024761759/Put/vlen=460950]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:131)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_8387547514055202675_51616042
> file=/hbase/orwell_events/a9906c96a91bb8d7e62a7a528bf0ea5c/U/d2ee47650b224189b0c27d1c20929c03
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      ... 17 more
> 2012-10-01 19:50:01,091 [IPC Server handler 24 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not iterate StoreFileScanner[HFileScanner
> for reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00318964/U:user/1349118541276/Put/vlen=311046]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:104)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_2851854722247682142_51616579
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      ... 14 more
> 2012-10-01 19:50:01,091 [IPC Server handler 1 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not iterate StoreFileScanner[HFileScanner
> for reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=0032027/U:KINGDOMSQUESTS_10/1349118531396/Put/vlen=401149]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:104)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_3201413024070455305_51616611
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      ... 14 more
> 2012-10-01 19:50:01,091 [IPC Server handler 25 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not iterate StoreFileScanner[HFileScanner
> for reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00319173/U:TINYTOWERANDROID_3/1349024232716/Put/vlen=129419]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:104)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_2851854722247682142_51616579
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      ... 14 more
> 2012-10-01 19:50:01,091 [IPC Server handler 90 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not iterate StoreFileScanner[HFileScanner
> for reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00316914/U:PETCAT_2/1349118542022/Put/vlen=499140]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:104)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_5937357897784147544_51616546
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      ... 14 more
> 2012-10-01 19:50:01,091 [IPC Server handler 17 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not seek StoreFileScanner[HFileScanner for
> reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00317054/U:BAHAMUTIOS_4/1348869430278/Put/vlen=104012]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:131)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_5937357897784147544_51616546
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:200)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      ... 17 more
> 2012-10-01 19:50:01,091 [IPC Server handler 58 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not iterate StoreFileScanner[HFileScanner
> for reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00316983/U:TINYTOWERANDROID_1/1349118439250/Put/vlen=417924]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:104)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:106)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:289)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_5937357897784147544_51616546
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readNextDataBlock(HFileReaderV2.java:452)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:416)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:99)
>      ... 14 more
> 2012-10-01 19:50:01,091 [IPC Server handler 89 on 60020] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> java.io.IOException: Could not seek StoreFileScanner[HFileScanner for
> reader reader=hdfs://namenode301.ngpipes.milp.ngmoco.com:9000/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false]
> [cacheCompressed=false],
> firstKey=00316914/U:PETCAT_1/1349118541277/Put,
> lastKey=00321083/U:user/1349024170056/Put, avgKeyLen=31,
> avgValueLen=89140, entries=7365, length=656954017,
> cur=00317043/U:BAHAMUTANDROID_7/1348968079952/Put/vlen=419212]
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:131)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap$SeekType$1.seek(KeyValueHeap.java:57)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:277)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:248)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:436)
>      at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:322)
>      at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:138)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:2945)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2901)
>      at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:2918)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3693)
>      at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:3585)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Could not obtain block:
> blk_5937357897784147544_51616546
> file=/hbase/orwell_events/9740f22a42e9e8b6aca3966c0173e680/U/2e5534fea8b24eaf9cc1e05dea788c01
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1993)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchBlockByteRange(DFSClient.java:2028)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2116)
>      at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:46)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1034)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1318)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:266)
>      at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:519)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.reseekTo(HFileReaderV2.java:557)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:194)
>      at org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:124)
>      ... 17 more
> 2012-10-01 19:50:01,094 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,094 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,093 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,093 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,092 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,092 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,091 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.ipc.Client: interrupted waiting to send params to
> server
> java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:01,095 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:50:01,097 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:50:01,115 [IPC Server handler 39 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@2743ecf8,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00390925"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.122:51758: output error
> 2012-10-01 19:50:01,139 [IPC Server handler 39 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 39 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,139 [IPC Server handler 39 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 39 on 60020:
> exiting
> 2012-10-01 19:50:01,151 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #1 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:01,151 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 2 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010. Will
> retry...
> 2012-10-01 19:50:01,153 [IPC Server handler 89 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@7137feec,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00317043"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.68:55302: output error
> 2012-10-01 19:50:01,154 [IPC Server handler 89 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 89 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,154 [IPC Server handler 89 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 89 on 60020:
> exiting
> 2012-10-01 19:50:01,156 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@6b9a9eba,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00321504"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.176:32793: output error
> 2012-10-01 19:50:01,157 [IPC Server handler 66 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 66 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,158 [IPC Server handler 66 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 66 on 60020:
> exiting
> 2012-10-01 19:50:01,159 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@586761c,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00391525"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.155:39850: output error
> 2012-10-01 19:50:01,160 [IPC Server handler 41 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 41 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,160 [IPC Server handler 41 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 41 on 60020:
> exiting
> 2012-10-01 19:50:01,216 [regionserver60020.compactionChecker] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer$CompactionChecker:
> regionserver60020.compactionChecker exiting
> 2012-10-01 19:50:01,216 [regionserver60020.logRoller] INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
> 2012-10-01 19:50:01,216 [regionserver60020.cacheFlusher] INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher:
> regionserver60020.cacheFlusher exiting
> 2012-10-01 19:50:01,217 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: aborting server
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272
> 2012-10-01 19:50:01,218 [regionserver60020] INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> Closed zookeeper sessionid=0x137ec64373dd4b3
> 2012-10-01 19:50:01,270
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,24294294,1349027918385.068e6f4f7b8a81fb21e49fe3ac47f262.
> 2012-10-01 19:50:01,271
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96510144,1348960969795.fe2a133a17d09a65a6b0d4fb60e6e051.
> 2012-10-01 19:50:01,272
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56499174,1349027424070.7f767ca333bef3dcdacc9a6c673a8350.
> 2012-10-01 19:50:01,273
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96515494,1348960969795.8ab4e1d9f4e4c388f3f4f18eec637e8a.
> 2012-10-01 19:50:01,273
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,98395724,1348969940123.08188cc246bf752c17cfe57f99970924.
> 2012-10-01 19:50:01,274
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56469014,1348940247961.6ace1f2f0b7ad3e454f738d66255047f.
> 2012-10-01 19:50:01,275
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56604984,1348940650040.14639a082062e98abfea8ae3fff5d2c7.
> 2012-10-01 19:50:01,275
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56880144,1348969971950.ece85a086a310aacc2da259a3303e67e.
> 2012-10-01 19:50:01,276
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56447305,1349027937173.fb85da01ca228de9f9ac6ffa63416e9b.
> 2012-10-01 19:50:01,277
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,31267284,1348961229728.fc429276c44f5c274f00168f12128bad.
> 2012-10-01 19:50:01,278
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56569824,1348940809479.9808dac5b895fc9b8f9892c4b72b3804.
> 2012-10-01 19:50:01,279
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56425354,1349031095620.e4965f2e57729ff9537986da3e19258c.
> 2012-10-01 19:50:01,280
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96504305,1348964001164.77f75cf8ba76ebc4417d49f019317d0a.
> 2012-10-01 19:50:01,280
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,60743825,1348962513777.f377f704db5f0d000e36003338e017b1.
> 2012-10-01 19:50:01,283
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,09603014,1349026790546.d634bfe659bdf2f45ec89e53d2d38791.
> 2012-10-01 19:50:01,283
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,31274021,1348961229728.e93382b458a84c22f2e5aeb9efa737b5.
> 2012-10-01 19:50:01,285
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56462454,1348982699951.a2dafbd054bf65aa6f558dc9a2d839a1.
> 2012-10-01 19:50:01,286
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> Orwell,48814673,1348270987327.29818ea19d62126d5616a7ba7d7dae21.
> 2012-10-01 19:50:01,288
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56610954,1348940650040.3609c1bfc2be6936577b6be493e7e8d9.
> 2012-10-01 19:50:01,289
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56511684,1349027965795.f0198c0a2a34f18da689910235a9b0e2.
> 2012-10-01 19:50:01,289
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,05205763,1348941089603.957ea0e428ba6ff21174ecdda96f9fdc.
> 2012-10-01 19:50:01,289
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56349615,1348941138879.dfabbd25c59fd6c34a58d9eacf4c096f.
> 2012-10-01 19:50:01,292
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56503505,1349027424070.129160a78f13c17cc9ea16ff3757cda9.
> 2012-10-01 19:50:01,292
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,91248264,1348942310344.a93982b8f91f260814885bc0afb4fbb9.
> 2012-10-01 19:50:01,293
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,98646724,1348980566403.a4f2a16d1278ad1246068646c4886502.
> 2012-10-01 19:50:01,293
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56454594,1348982903997.7107c6a1b2117fb59f68210ce82f2cc9.
> 2012-10-01 19:50:01,294
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56564144,1348940809479.636092bb3ec2615b115257080427d091.
> 2012-10-01 19:50:01,295
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_user_events,06252594,1348582793143.499f0a0f4704afa873c83f141f5e0324.
> 2012-10-01 19:50:01,296
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56617164,1348941287729.3992a80a6648ab62753b4998331dcfdf.
> 2012-10-01 19:50:01,296
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,98390944,1348969940123.af160e450632411818fa8d01b2c2ed0b.
> 2012-10-01 19:50:01,297
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56703743,1348941223663.5cc2fcb82080dbf14956466c31f1d27c.
> 2012-10-01 19:50:01,297
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56427793,1349031095620.88fa75b4719f4b83b9165474139c4a94.
> 2012-10-01 19:50:01,298
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56693584,1348942631318.f01b179c1fad1f18b97b37fc8f730898.
> 2012-10-01 19:50:01,299
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_user_events,12140615,1348582250428.7822f7f5ceea852b04b586fdf34debff.
> 2012-10-01 19:50:01,300
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56388374,1348941251489.8c5f4903b82a4ff64ff1638c95692b60.
> 2012-10-01 19:50:01,300
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96420705,1348942597601.a063e06eb840ee49bb88474ee8e22160.
> 2012-10-01 19:50:01,300
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56414764,1349027912141.74a852e5b4186edd51ca714bd77f80c0.
> 2012-10-01 19:50:01,300
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96432674,1348961425148.1a793cf2137b9599193a1e2d5d9749c5.
> 2012-10-01 19:50:01,302
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56329744,1349028887914.9686d8348fe53644334c0423cc217d26.
> 2012-10-01 19:50:01,303
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,44371574,1348961840615.00f5b4710a43f2ee75d324bebb054323.
> 2012-10-01 19:50:01,304
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,562fc921,1348941189517.cff261c585416844113f232960c8d6b4.
> 2012-10-01 19:50:01,304
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56323831,1348941216581.0b0f3bdb03ce9e4f58156a4143018e0e.
> 2012-10-01 19:50:01,305
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56480194,1349028080664.03a7046ffcec7e1f19cdb2f9890a353e.
> 2012-10-01 19:50:01,306
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56418294,1348940288044.c872be05981c047e8c1ee4765b92a74d.
> 2012-10-01 19:50:01,306
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,53590305,1348940776419.4c98d7846622f2d8dad4e998dae81d2b.
> 2012-10-01 19:50:01,307
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96445963,1348942353563.66a0f602720191bf21a1dfd12eec4a35.
> 2012-10-01 19:50:01,307
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,5649502,1349027928183.2088f23f8fb1dbc67b972f8744aca289.
> 2012-10-01 19:50:01,307
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56305294,1348941189517.20f67941294c259e2273d3e0b7ae5198.
> 2012-10-01 19:50:01,308
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56516115,1348981132325.0f753cb87c1163d95d9d10077d6308db.
> 2012-10-01 19:50:01,309
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56796924,1348941269761.843e0aee0b15d67b810c7b3fe5a2dda7.
> 2012-10-01 19:50:01,309
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56440004,1348941150045.7033cb81a66e405d7bf45cd55ab010e3.
> 2012-10-01 19:50:01,309
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56317864,1348941124299.0de45283aa626fc83b2c026e1dd8bfec.
> 2012-10-01 19:50:01,310
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56809673,1348941834500.08244d4ed5f7fdf6d9ac9c73fbfd3947.
> 2012-10-01 19:50:01,310
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56894864,1348970959541.fc19a6ffe18f29203369d32ad1b102ce.
> 2012-10-01 19:50:01,311
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56382491,1348940876960.2392137bf0f4cb695c08c0fb22ce5294.
> 2012-10-01 19:50:01,312
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,95128264,1349026585563.5dc569af8afe0a84006b80612c15007f.
> 2012-10-01 19:50:01,312
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,5631146,1348941124299.b7c10be9855b5e8ba3a76852920627f9.
> 2012-10-01 19:50:01,312
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56710424,1348940462668.a370c149c232ebf4427e070eb28079bc.
> 2012-10-01 19:50:01,314 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Session: 0x137ec64373dd4b3 closed
> 2012-10-01 19:50:01,314 [regionserver60020-EventThread] INFO
> org.apache.zookeeper.ClientCnxn: EventThread shut down
> 2012-10-01 19:50:01,314 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on 78
> regions to close
> 2012-10-01 19:50:01,317
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96497834,1348964001164.0b12f37b74b2124ef9f27d1ef0ebb17a.
> 2012-10-01 19:50:01,318
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56507574,1349027965795.79113c51d318a11286b39397ebbfdf04.
> 2012-10-01 19:50:01,319
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,24297525,1349027918385.047533f3d801709a26c895a01dcc1a73.
> 2012-10-01 19:50:01,320
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96439694,1348961425148.038e0e43a6e56760e4daae6f34bfc607.
> 2012-10-01 19:50:01,320
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,82811715,1348904784424.88fae4279f9806bef745d90f7ad37241.
> 2012-10-01 19:50:01,321
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56699434,1348941223663.ef3ccf0af60ee87450806b393f89cb6e.
> 2012-10-01 19:50:01,321
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56487344,1349027988535.cd7e7eb88967b3dcb223de9c4ad807a9.
> 2012-10-01 19:50:01,322
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56601774,1349029123008.34b52e7208034f85db8d1e39ca6c1329.
> 2012-10-01 19:50:01,322
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56586234,1349028973106.44449344385d98cd7512008dfa532f8e.
> 2012-10-01 19:50:01,323
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56465563,1348982699951.f34a29c0c4fc32e753d12db996ccc995.
> 2012-10-01 19:50:01,324
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56450734,1349027937173.c70110b3573a48299853117c4287c7be.
> 2012-10-01 19:50:01,325
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56361984,1349029457686.6c8d6974741e59df971da91c7355de1c.
> 2012-10-01 19:50:01,327
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56814705,1348962077056.69fd74167a3c5c2961e45d339b962ca9.
> 2012-10-01 19:50:01,327
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,00389105,1348978080963.6463149a16179d4e44c19bb49e4b4a81.
> 2012-10-01 19:50:01,329
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56558944,1348940893836.03bd1c0532949ec115ca8d5215dbb22f.
> 2012-10-01 19:50:01,330 [IPC Server handler 59 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@112ba2bf,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00392783"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.135:34935: output error
> 2012-10-01 19:50:01,330
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,5658955,1349027142822.e65d0c1f452cb41d47ad08560c653607.
> 2012-10-01 19:50:01,331 [IPC Server handler 59 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 59 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:01,331
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56402364,1349049689267.27b452f3bcce0815b7bf92370cbb51de.
> 2012-10-01 19:50:01,331 [IPC Server handler 59 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 59 on 60020:
> exiting
> 2012-10-01 19:50:01,332
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96426544,1348942597601.addf704f99dd1b2e07b3eff505e2c811.
> 2012-10-01 19:50:01,333
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,60414161,1348962852909.c6b1b21f00bbeef8648c4b9b3d28b49a.
> 2012-10-01 19:50:01,333
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56552794,1348940893836.5314886f88f6576e127757faa25cef7c.
> 2012-10-01 19:50:01,335
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56910924,1348962040261.fdedae86206fc091a72dde52a3d0d0b4.
> 2012-10-01 19:50:01,335
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56720084,1349029064698.ee5cb00ab358be0d2d36c59189da32f8.
> 2012-10-01 19:50:01,336
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56624533,1348941287729.6121fce2c31d4754b4ad4e855d85b501.
> 2012-10-01 19:50:01,336
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56899934,1348970959541.f34f01dd65e293cb6ab13de17ac91eec.
> 2012-10-01 19:50:01,337
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56394773,1348941251489.f81d0498ab42b400f37a48d4f3854006.
> 2012-10-01 19:50:01,337
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56405923,1349049689267.bb4be5396608abeff803400cdd2408f4.
> 2012-10-01 19:50:01,338
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56364924,1349029457686.1e1c09b6eb734d8ad48ea0b4fa103381.
> 2012-10-01 19:50:01,339
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56784073,1348961864297.f01eaf712e59a0bca989ced951caf4f1.
> 2012-10-01 19:50:01,340
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56594534,1349027142822.8e67bb85f4906d579d4d278d55efce0b.
> 2012-10-01 19:50:01,340
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56342344,1348941231573.8adf268d4fcb494344745c14b090e773.
> 2012-10-01 19:50:01,340
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56491525,1349027928183.7bbfb4d39ef4332e17845001191a6ad4.
> 2012-10-01 19:50:01,341
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,07123624,1348959804638.c114ec80c6693a284741e220da028736.
> 2012-10-01 19:50:01,342
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56538694,1348941049708.b9f56cf1f6f6c7b0cdf2a07a3d36846b.
> 2012-10-01 19:50:01,342
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56546534,1348941049708.bde2614732f938db04fdd81ed6dbfcf2.
> 2012-10-01 19:50:01,343
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,569054,1348962040261.a7942d7837cd57b68d156d2ce7e3bd5f.
> 2012-10-01 19:50:01,343
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56576714,1348982931576.3dd5bf244fb116cf2b6f812fcc39ad2d.
> 2012-10-01 19:50:01,344
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,5689007,1348963034009.c4b16ea4d8dbc66c301e67d8e58a7e48.
> 2012-10-01 19:50:01,344
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56410784,1349027912141.6de7be1745c329cf9680ad15e9bde594.
> 2012-10-01 19:50:01,345
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56332944,1349028887914.506f6865d3167d722fec947a59761822.
> 2012-10-01 19:50:01,345
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96457954,1348964300132.674a03f0c9866968aabd70ab38a482c0.
> 2012-10-01 19:50:01,346
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56483084,1349027988535.de732d7e63ea53331b80255f51fc1a86.
> 2012-10-01 19:50:01,347
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56790484,1348941269761.5bcc58c48351de449cc17307ab4bf777.
> 2012-10-01 19:50:01,348
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56458293,1348982903997.4f67e6f4949a2ef7f4903f78f54c474e.
> 2012-10-01 19:50:01,348
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,95123235,1349026585563.a359eb4cb88d34a529804e50a5affa24.
> 2012-10-01 19:50:01,349
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56597943,1349029123008.bf64051a387fc2970252a1c8919dfd88.
> 2012-10-01 19:50:01,350
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56368484,1348941099873.cef2729093a0d7d72b71fac1b25c0a40.
> 2012-10-01 19:50:01,350
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,17499894,1349026916228.630196a553f73069b9e568e6912ef0c5.
> 2012-10-01 19:50:01,351
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56375315,1348940876960.40cf6dfa370ce7f1fc6c1a59ba2f2191.
> 2012-10-01 19:50:01,351
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,95512574,1349009451986.e4d292eb66d16c21ef8ae32254334850.
> 2012-10-01 19:50:01,352
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56717254,1349029064698.31a1dcefef4d5e3133b323cdaac918d7.
> 2012-10-01 19:50:01,352
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56532464,1348941298909.31c8b60bb6ad6840de937a28e3482101.
> 2012-10-01 19:50:01,353
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56432705,1348941150045.07aa626f3703c7b4deaba1263c71894d.
> 2012-10-01 19:50:01,353
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,13118725,1349026772953.c0be859d4a4dc2246d764a8aad58fe88.
> 2012-10-01 19:50:01,354
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56520814,1348981132325.c2f16fd16f83aa51769abedfe8968bb6.
> 2012-10-01 19:50:01,354
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,95507615,1349009451986.c08d16db6188bd8cec100eeb1291d5b9.
> 2012-10-01 19:50:01,355
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56884434,1348963034009.616835869c81659a27eab896f48ae4e1.
> 2012-10-01 19:50:01,355
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56476541,1349028080664.341392a325646f24a3d8b8cd27ebda19.
> 2012-10-01 19:50:01,357
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56803462,1348941834500.6313b36f1949381d01df977a182e6140.
> 2012-10-01 19:50:01,357
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96464524,1348964300132.7a15f1e8e28f713212c516777267c2bf.
> 2012-10-01 19:50:01,358
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56875074,1348969971950.3e408e7cb32c9213d184e10bf42837ad.
> 2012-10-01 19:50:01,359
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,42862354,1348981565262.7ad46818060be413140cdcc11312119d.
> 2012-10-01 19:50:01,359
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed

> orwell_events,56582264,1349028973106.b481b61be387a041a3f259069d5013a6.
> 2012-10-01 19:50:01,360
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56579105,1348982931576.1561a22c16263dccb8be07c654b43f2f.
> 2012-10-01 19:50:01,360
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56723415,1348946404223.38d992d687ad8925810be4220a732b13.
> 2012-10-01 19:50:01,361
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,4285921,1348981565262.7a2cbd8452b9e406eaf1a5ebff64855a.
> 2012-10-01 19:50:01,362
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56336394,1348941231573.ca52393a2eabae00a64f65c0b657b95a.
> 2012-10-01 19:50:01,363
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,96452715,1348942353563.876edfc6e978879aac42bfc905a09c26.
> 2012-10-01 19:50:01,363
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56355104,1348941138879.326405233b2b444691860b14ef587f78.
> 2012-10-01 19:50:01,364
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56525625,1348941298909.ccf16ed8e761765d2989343c7670e94f.
> 2012-10-01 19:50:01,365
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,97578484,1348938848996.98ecacc61ae4c5b3f7a3de64bec0e026.
> 2012-10-01 19:50:01,365
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56779025,1348961864297.cc13f0a6f5e632508f2e28a174ef1488.
> 2012-10-01 19:50:01,366
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56728994,1348946404223.99012cf45da4109e6b570e8b0178852c.
> 2012-10-01 19:50:01,366
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_user_events,43323443,1348591057882.8b0ab02c33f275114d89088345f58885.
> 2012-10-01 19:50:01,367
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56819704,1348962077056.621ebefbdb194a82d6314ff0f58b67b1.
> 2012-10-01 19:50:01,367
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,56686234,1348942631318.69270cd5013f8ca984424e508878e428.
> 2012-10-01 19:50:01,368
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,98642625,1348980566403.2277d2ef1d53d40d41cd23846619a3f8.
> 2012-10-01 19:50:01,524 [IPC Server handler 57 on 60020] INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_3201413024070455305_51616611 from any node: java.io.IOException:
> No live nodes contain current block. Will get new block locations from
> namenode and retry...
> 2012-10-01 19:50:02,462 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on 2
> regions to close
> 2012-10-01 19:50:02,462 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:50:02,462 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:50:02,495 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #2 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:02,496 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 3 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010. Will
> retry...
> 2012-10-01 19:50:02,686 [IPC Server handler 46 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@504b62c6,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00320404"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.172:53925: output error
> 2012-10-01 19:50:02,688 [IPC Server handler 46 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 46 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:02,688 [IPC Server handler 46 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 46 on 60020:
> exiting
> 2012-10-01 19:50:02,809 [IPC Server handler 55 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@45f1c31e,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00322424"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.178:35016: output error
> 2012-10-01 19:50:02,810 [IPC Server handler 55 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 55 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:02,810 [IPC Server handler 55 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 55 on 60020:
> exiting
> 2012-10-01 19:50:03,496 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:50:03,496 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:50:03,510 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #3 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:03,510 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 4 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010. Will
> retry...
> 2012-10-01 19:50:05,299 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:50:05,299 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:50:05,314 [IPC Server handler 3 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@472aa9fe,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00321694"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.176:42371: output error
> 2012-10-01 19:50:05,315 [IPC Server handler 3 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:05,315 [IPC Server handler 3 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60020:
> exiting
> 2012-10-01 19:50:05,329 [IPC Server handler 16 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@42987a12,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00320293"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.135:35132: output error
> 2012-10-01 19:50:05,331 [IPC Server handler 16 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 16 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:05,331 [IPC Server handler 16 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 16 on 60020:
> exiting
> 2012-10-01 19:50:05,638 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #4 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:05,638 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 5 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010. Will
> retry...
> 2012-10-01 19:50:05,641 [IPC Server handler 26 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@a9c09e8,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00319505"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.183:60078: output error
> 2012-10-01 19:50:05,643 [IPC Server handler 26 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 26 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:05,643 [IPC Server handler 26 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 26 on 60020:
> exiting
> 2012-10-01 19:50:05,664 [IPC Server handler 57 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@349d7b4,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00319915"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.141:58290: output error
> 2012-10-01 19:50:05,666 [IPC Server handler 57 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 57 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:05,666 [IPC Server handler 57 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 57 on 60020:
> exiting
> 2012-10-01 19:50:07,063 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 bad datanode[0] 10.100.101.156:50010
> 2012-10-01 19:50:07,063 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 in pipeline 10.100.101.156:50010,
> 10.100.102.88:50010, 10.100.102.122:50010: bad datanode
> 10.100.101.156:50010
> 2012-10-01 19:50:07,076 [IPC Server handler 23 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@5ba03734,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00319654"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.161:43227: output error
> 2012-10-01 19:50:07,077 [IPC Server handler 23 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 23 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:07,077 [IPC Server handler 23 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 23 on 60020:
> exiting
> 2012-10-01 19:50:07,089 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #5 from
> primary datanode 10.100.102.122:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:07,090 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.122:50010 failed 6 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010, 10.100.102.122:50010.
> Marking primary datanode as bad.
> 2012-10-01 19:50:07,173 [IPC Server handler 85 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@3d19e607,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00319564"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.82:42779: output error
> 2012-10-01 19:50:07,173 [IPC Server handler 85 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 85 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:07,173 [IPC Server handler 85 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 85 on 60020:
> exiting
> 2012-10-01 19:50:07,181
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-2]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,00321084,1349118541283.a9906c96a91bb8d7e62a7a528bf0ea5c.
> 2012-10-01 19:50:07,693 [IPC Server handler 79 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@5920511b,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00322014"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.88:49489: output error
> 2012-10-01 19:50:07,693 [IPC Server handler 79 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 79 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:07,693 [IPC Server handler 79 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 79 on 60020:
> exiting
> 2012-10-01 19:50:08,064 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on 1
> regions to close
> 2012-10-01 19:50:08,159 [regionserver60020.leaseChecker] INFO
> org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020.leaseChecker closing leases
> 2012-10-01 19:50:08,159 [regionserver60020.leaseChecker] INFO
> org.apache.hadoop.hbase.regionserver.Leases:
> regionserver60020.leaseChecker closed leases
> 2012-10-01 19:50:08,508 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #0 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:08,508 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 1 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:09,652 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #1 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:09,653 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 2 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:10,697 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #2 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:10,697 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 3 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:12,278 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #3 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:12,279 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 4 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:13,294 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #4 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:13,294 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 5 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:14,306 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #5 from
> primary datanode 10.100.101.156:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:14,306 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.101.156:50010 failed 6 times.  Pipeline was
> 10.100.101.156:50010, 10.100.102.88:50010. Marking primary datanode as
> bad.
> 2012-10-01 19:50:15,317 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #0 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:15,318 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 1 times.  Pipeline was
> 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:16,375 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #1 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:16,376 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 2 times.  Pipeline was
> 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:17,385 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #2 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:17,385 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 3 times.  Pipeline was
> 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:18,395 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #3 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:18,395 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 4 times.  Pipeline was
> 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:19,404 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #4 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:19,405 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 5 times.  Pipeline was
> 10.100.102.88:50010. Will retry...
> 2012-10-01 19:50:20,414 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Failed recovery attempt #5 from
> primary datanode 10.100.102.88:50010
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> blk_5535637699691880681_51616301 is already commited, storedBlock ==
> null.
>      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:5348)
>      at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:705)
>      at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy4.nextGenerationStamp(Unknown Source)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1886)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1854)
>      at org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1950)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
>      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at javax.security.auth.Subject.doAs(Subject.java:396)
>      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
>
>      at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy14.recoverBlock(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2793)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,415 [DataStreamer for file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> block blk_5535637699691880681_51616301] WARN
> org.apache.hadoop.hdfs.DFSClient: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
> 2012-10-01 19:50:20,415 [IPC Server handler 58 on 60020] ERROR
> org.apache.hadoop.hdfs.DFSClient: Exception closing file
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1341428844272.1349118568164
> : java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,415 [IPC Server handler 69 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,415 [regionserver60020.logSyncer] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,417 [IPC Server handler 69 on 60020] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,417 [IPC Server handler 58 on 60020] INFO
> org.apache.hadoop.fs.FileSystem: Could not cancel cleanup thread,
> though no FileSystems are open
> 2012-10-01 19:50:20,417 [regionserver60020.logSyncer] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,417 [regionserver60020.logSyncer] FATAL
> org.apache.hadoop.hbase.regionserver.wal.HLog: Could not sync.
> Requesting close of hlog
> java.io.IOException: Reflection
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:230)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1098)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1202)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1060)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
>      at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:228)
>      ... 4 more
> Caused by: java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,418 [regionserver60020.logSyncer] ERROR
> org.apache.hadoop.hbase.regionserver.wal.HLog: Error while syncing,
> requesting close of hlog
> java.io.IOException: Reflection
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:230)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1098)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1202)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1060)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
>      at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:228)
>      ... 4 more
> Caused by: java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,417 [IPC Server handler 69 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.wal.HLog: Could not sync.
> Requesting close of hlog
> java.io.IOException: Reflection
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:230)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1098)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1202)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.append(HLog.java:1033)
>      at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchPut(HRegion.java:1852)
>      at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1723)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3076)
>      at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.lang.reflect.InvocationTargetException
>      at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:228)
>      ... 11 more
> Caused by: java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:20,417 [IPC Server handler 29 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,417 [IPC Server handler 24 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,420 [IPC Server handler 24 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,417 [IPC Server handler 1 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,421 [IPC Server handler 1 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,417 [IPC Server handler 25 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,421 [IPC Server handler 25 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,417 [IPC Server handler 90 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,422 [IPC Server handler 90 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,417 [IPC Server handler 58 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,423 [IPC Server handler 58 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,417 [IPC Server handler 17 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
> server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272: File
> System not available
> java.io.IOException: File system is not available
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:146)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1122)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: java.lang.InterruptedException
>      at org.apache.hadoop.ipc.Client.call(Client.java:1086)
>      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>      at $Proxy7.getFileInfo(Unknown Source)
>      at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:697)
>      at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542)
>      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748)
>      at org.apache.hadoop.hbase.util.FSUtils.checkFileSystemAvailable(FSUtils.java:135)
>      ... 9 more
> Caused by: java.lang.InterruptedException
>      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1279)
>      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>      at org.apache.hadoop.ipc.Client$Connection.sendParam(Client.java:790)
>      at org.apache.hadoop.ipc.Client.call(Client.java:1080)
>      ... 21 more
> 2012-10-01 19:50:20,423 [IPC Server handler 17 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,423 [IPC Server handler 58 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1576,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,423 [IPC Server handler 17 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,

> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1576,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,422 [IPC Server handler 90 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1576,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,421 [IPC Server handler 25 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1576,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,421 [IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1576,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,420 [IPC Server handler 69 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: (responseTooSlow):
> {"processingtimems":22039,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@207c46fb),
> rpc version=1, client version=29,
> methodsFingerPrint=54742778","client":"10.100.102.155:39852","starttimems":1349120998380,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
> 2012-10-01 19:50:20,420 [IPC Server handler 24 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1575,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,420
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING
> region server data3024.ngpipes.milp.ngmoco.com,60020,1341428844272:
> Unrecoverable exception while closing region
> orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.,
> still finishing close
> java.io.IOException: Filesystem closed
>      at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:232)
>      at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:70)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:1859)
>      at java.io.FilterInputStream.close(FilterInputStream.java:155)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:320)
>      at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1081)
>      at org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:568)
>      at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:473)
>      at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:789)
>      at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:724)
>      at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
>      at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>      at java.lang.Thread.run(Thread.java:662)
> 2012-10-01 19:50:20,426
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,419 [IPC Server handler 29 on 60020] FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer
> abort: loaded coprocessors are: []
> 2012-10-01 19:50:20,426
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of
> metrics: requestsPerSecond=0, numberOfOnlineRegions=136,
> numberOfStores=136, numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1577,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,426 [IPC Server handler 29 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> requestsPerSecond=0, numberOfOnlineRegions=136, numberOfStores=136,
> numberOfStorefiles=189, storefileIndexSizeMB=15,
> rootIndexSizeKB=16019, totalStaticIndexSizeKB=16194,
> totalStaticBloomSizeKB=0, memstoreSizeMB=113,
> readRequestsCount=6744201, writeRequestsCount=904280,
> compactionQueueSize=0, flushQueueSize=0, usedHeapMB=1577,
> maxHeapMB=3987, blockCacheSizeMB=781.51, blockCacheFreeMB=215.3,
> blockCacheCount=5435, blockCacheHitCount=321294212,
> blockCacheMissCount=4657926, blockCacheEvictedCount=1864312,
> blockCacheHitRatio=98%, blockCacheHitCachingRatio=99%,
> hdfsBlocksLocalityIndex=97
> 2012-10-01 19:50:20,445 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedChannelException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.FilterInputStream.read(FilterInputStream.java:116)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> 2012-10-01 19:50:20,446 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedByInterruptException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
>      at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
>      at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
>      at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>      at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>      at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:545)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:898)
>      ... 12 more
> 2012-10-01 19:50:20,447 [IPC Server handler 29 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,446 [IPC Server handler 58 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,446 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:511)
>      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
>      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> 2012-10-01 19:50:20,448 [IPC Server handler 17 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,445 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedChannelException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.FilterInputStream.read(FilterInputStream.java:116)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> 2012-10-01 19:50:20,448 [IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,445
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> WARN org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to
> report fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:131)
>      at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedByInterruptException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 7 more
> Caused by: java.nio.channels.ClosedByInterruptException
>      at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
>      at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
>      at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
>      at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>      at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>      at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:545)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:898)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> 2012-10-01 19:50:20,450
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> Unrecoverable exception while closing region
> orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.,
> still finishing close
> 2012-10-01 19:50:20,445 [IPC Server handler 69 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> multi(org.apache.hadoop.hbase.client.MultiAction@207c46fb), rpc
> version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.155:39852: output error
> 2012-10-01 19:50:20,445 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedChannelException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.FilterInputStream.read(FilterInputStream.java:116)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> 2012-10-01 19:50:20,451 [IPC Server handler 24 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,445 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedChannelException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.FilterInputStream.read(FilterInputStream.java:116)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> 2012-10-01 19:50:20,451 [IPC Server handler 90 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,445 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to report
> fatal error to master
> java.lang.reflect.UndeclaredThrowableException
>      at $Proxy8.reportRSFatalError(Unknown Source)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1564)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.checkFileSystem(HRegionServer.java:1124)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1068)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.cleanup(HRegionServer.java:1043)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1787)
>      at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> Caused by: java.io.IOException: Call to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:60000 failed on
> local exception: java.nio.channels.ClosedChannelException
>      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:953)
>      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:922)
>      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>      ... 11 more
> Caused by: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:120)
>      at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:163)
>      at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
>      at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>      at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>      at java.io.FilterInputStream.read(FilterInputStream.java:116)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:311)
>      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>      at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>      at java.io.DataInputStream.readInt(DataInputStream.java:370)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:571)
>      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
> 2012-10-01 19:50:20,452 [IPC Server handler 25 on 60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: File
> System not available
> 2012-10-01 19:50:20,452 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@5d72e577,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00321312"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.184:34111: output error
> 2012-10-01 19:50:20,452 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@2237178f,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00316983"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.188:59581: output error
> 2012-10-01 19:50:20,450 [IPC Server handler 69 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 69 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelIO(HBaseServer.java:1710)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,452 [IPC Server handler 58 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 58 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,452 [IPC Server handler 58 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 58 on 60020:
> exiting
> 2012-10-01 19:50:20,450
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-1]
> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
> while processing event M_RS_CLOSE_REGION
> java.lang.RuntimeException: java.io.IOException: Filesystem closed
>      at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
>      at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Filesystem closed
>      at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:232)
>      at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:70)
>      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:1859)
>      at java.io.FilterInputStream.close(FilterInputStream.java:155)
>      at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:320)
>      at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.close(StoreFile.java:1081)
>      at org.apache.hadoop.hbase.regionserver.StoreFile.closeReader(StoreFile.java:568)
>      at org.apache.hadoop.hbase.regionserver.Store.close(Store.java:473)
>      at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:789)
>      at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:724)
>      at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
>      ... 4 more
> 2012-10-01 19:50:20,453 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@573dba6d,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"0032027"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.183:60076: output error
> 2012-10-01 19:50:20,452 [IPC Server handler 69 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 69 on 60020:
> exiting
> 2012-10-01 19:50:20,453 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@4eebbed5,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00317054"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.146:40240: output error
> 2012-10-01 19:50:20,452 [IPC Server handler 29 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 29 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,453 [IPC Server handler 29 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 29 on 60020:
> exiting
> 2012-10-01 19:50:20,453 [IPC Server handler 1 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,453 [IPC Server handler 17 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 17 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,453 [IPC Server handler 17 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 17 on 60020:
> exiting
> 2012-10-01 19:50:20,453 [IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020:
> exiting
> 2012-10-01 19:50:20,454 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@4ff0ed4a,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00318964"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.172:53924: output error
> 2012-10-01 19:50:20,454 [IPC Server handler 24 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 24 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,454 [IPC Server handler 24 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 24 on 60020:
> exiting
> 2012-10-01 19:50:20,455 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@526abe46,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00316914"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.101.184:34110: output error
> 2012-10-01 19:50:20,455 [IPC Server handler 90 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 90 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,455 [IPC Server handler 90 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 90 on 60020:
> exiting
> 2012-10-01 19:50:20,456 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
> get([B@5df20fef,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"U":["ALL"]},"maxVersions":1,"row":"00319173"}),
> rpc version=1, client version=29, methodsFingerPrint=54742778 from
> 10.100.102.146:40243: output error
> 2012-10-01 19:50:20,456 [IPC Server handler 25 on 60020] WARN
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 25 on 60020
> caught: java.nio.channels.ClosedChannelException
>      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
>      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>      at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1653)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:924)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:1003)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:409)
>      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1346)
>
> 2012-10-01 19:50:20,456 [IPC Server handler 25 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 25 on 60020:
> exiting
> 2012-10-01 19:50:21,066
> [RS_CLOSE_REGION-data3024.ngpipes.milp.ngmoco.com,60020,1341428844272-0]
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed
> orwell_events,00316914,1349118541283.9740f22a42e9e8b6aca3966c0173e680.
> 2012-10-01 19:50:21,418 [regionserver60020.logSyncer] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:21,418 [regionserver60020.logSyncer] WARN
> org.apache.hadoop.hdfs.DFSClient: Error while syncing
> java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:21,418 [regionserver60020.logSyncer] FATAL
> org.apache.hadoop.hbase.regionserver.wal.HLog: Could not sync.
> Requesting close of hlog
> java.io.IOException: Reflection
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:230)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1098)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1202)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1060)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
>      at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:228)
>      ... 4 more
> Caused by: java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:21,419 [regionserver60020.logSyncer] ERROR
> org.apache.hadoop.hbase.regionserver.wal.HLog: Error while syncing,
> requesting close of hlog
> java.io.IOException: Reflection
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:230)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1098)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1202)
>      at org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.run(HLog.java:1060)
>      at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
>      at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.sync(SequenceFileLogWriter.java:228)
>      ... 4 more
> Caused by: java.io.IOException: Error Recovery for block
> blk_5535637699691880681_51616301 failed  because recovery from primary
> datanode 10.100.102.88:50010 failed 6 times.  Pipeline was
> 10.100.102.88:50010. Aborting...
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2833)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:2305)
>      at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2477)
> 2012-10-01 19:50:22,066 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272; all regions
> closed.
> 2012-10-01 19:50:22,066 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closing
> leases
> 2012-10-01 19:50:22,066 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.Leases: regionserver60020 closed
> leases
> 2012-10-01 19:50:22,082 [regionserver60020] WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed deleting my
> ephemeral node
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for
> /hbase/rs/data3024.ngpipes.milp.ngmoco.com,60020,1341428844272
>      at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
>      at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>      at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:868)
>      at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:107)
>      at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:962)
>      at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:951)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:964)
>      at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:762)
>      at java.lang.Thread.run(Thread.java:662)
> 2012-10-01 19:50:22,082 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server
> data3024.ngpipes.milp.ngmoco.com,60020,1341428844272; zookeeper
> connection closed.
> 2012-10-01 19:50:22,082 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: regionserver60020
> exiting
> 2012-10-01 19:50:22,123 [Shutdownhook:regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook
> starting; hbase.shutdown.hook=true;
> fsShutdownHook=Thread[Thread-5,5,main]
> 2012-10-01 19:50:22,123 [Shutdownhook:regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Shutdown
> hook
> 2012-10-01 19:50:22,123 [Shutdownhook:regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs
> shutdown hook thread.
> 2012-10-01 19:50:22,124 [Shutdownhook:regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook
> finished.
> Mon Oct  1 19:54:10 UTC 2012 Starting regionserver on
> data3024.ngpipes.milp.ngmoco.com
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 20
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 16382
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 32768
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) unlimited
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
> 2012-10-01 19:54:11,355 [main] INFO
> org.apache.hadoop.hbase.util.VersionInfo: HBase 0.92.1
> 2012-10-01 19:54:11,356 [main] INFO
> org.apache.hadoop.hbase.util.VersionInfo: Subversion
> https://svn.apache.org/repos/asf/hbase/branches/0.92 -r 1298924
> 2012-10-01 19:54:11,356 [main] INFO
> org.apache.hadoop.hbase.util.VersionInfo: Compiled by jenkins on Fri
> Mar  9 16:58:34 UTC 2012
> 2012-10-01 19:54:11,513 [main] INFO
> org.apache.hadoop.hbase.util.ServerCommandLine: vmName=Java
> HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc.,
> vmVersion=20.1-b02
> 2012-10-01 19:54:11,513 [main] INFO
> org.apache.hadoop.hbase.util.ServerCommandLine:
> vmInputArguments=[-XX:OnOutOfMemoryError=kill, -9, %p, -Xmx4000m,
> -XX:NewSize=128m, -XX:MaxNewSize=128m,
> -XX:+HeapDumpOnOutOfMemoryError, -XX:+UseConcMarkSweepGC,
> -XX:CMSInitiatingOccupancyFraction=75, -verbose:gc,
> -XX:+PrintGCDetails, -XX:+PrintGCTimeStamps,
> -Xloggc:/data2/hbase_log/gc-hbase.log,
> -Dcom.sun.management.jmxremote.authenticate=true,
> -Dcom.sun.management.jmxremote.ssl=false,
> -Dcom.sun.management.jmxremote.password.file=/home/hadoop/hadoop/conf/jmxremote.password,
> -Dcom.sun.management.jmxremote.port=8010,
> -Dhbase.log.dir=/data2/hbase_log,
> -Dhbase.log.file=hbase-hadoop-regionserver-data3024.ngpipes.milp.ngmoco.com.log,
> -Dhbase.home.dir=/home/hadoop/hbase, -Dhbase.id.str=hadoop,
> -Dhbase.root.logger=INFO,DRFA,
> -Djava.library.path=/home/hadoop/hbase/lib/native/Linux-amd64-64]
> 2012-10-01 19:54:11,964 [IPC Reader 0 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,967 [IPC Reader 1 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,970 [IPC Reader 2 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,973 [IPC Reader 3 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,976 [IPC Reader 4 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,979 [IPC Reader 5 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,982 [IPC Reader 6 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,985 [IPC Reader 7 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,988 [IPC Reader 8 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:11,991 [IPC Reader 9 on port 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
> 2012-10-01 19:54:12,002 [main] INFO
> org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics
> with hostName=HRegionServer, port=60020
> 2012-10-01 19:54:12,081 [main] INFO
> org.apache.hadoop.hbase.io.hfile.CacheConfig: Allocating LruBlockCache
> with maximum size 996.8m
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
> GMT
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:host.name=data3024.ngpipes.milp.ngmoco.com
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_26
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun
> Microsystems Inc.
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.26/jre
> 2012-10-01 19:54:12,221 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:java.class.path=/home/hadoop/hbase/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop/hbase:/home/hadoop/hbase/hbase-0.92.1.jar:/home/hadoop/hbase/hbase-0.92.1-tests.jar:/home/hadoop/hbase/lib/activation-1.1.jar:/home/hadoop/hbase/lib/asm-3.1.jar:/home/hadoop/hbase/lib/avro-1.5.3.jar:/home/hadoop/hbase/lib/avro-ipc-1.5.3.jar:/home/hadoop/hbase/lib/commons-beanutils-1.7.0.jar:/home/hadoop/hbase/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hbase/lib/commons-cli-1.2.jar:/home/hadoop/hbase/lib/commons-codec-1.4.jar:/home/hadoop/hbase/lib/commons-collections-3.2.1.jar:/home/hadoop/hbase/lib/commons-configuration-1.6.jar:/home/hadoop/hbase/lib/commons-digester-1.8.jar:/home/hadoop/hbase/lib/commons-el-1.0.jar:/home/hadoop/hbase/lib/commons-httpclient-3.1.jar:/home/hadoop/hbase/lib/commons-lang-2.5.jar:/home/hadoop/hbase/lib/commons-logging-1.1.1.jar:/home/hadoop/hbase/lib/commons-math-2.1.jar:/home/hadoop/hbase/lib/commons-net-1.4.1.jar:/home/hadoop/hbase/lib/core-3.1.1.jar:/home/hadoop/hbase/lib/guava-r09.jar:/home/hadoop/hbase/lib/hadoop-core-0.20.2-cdh3u2.jar:/home/hadoop/hbase/lib/hadoop-lzo-0.4.9.jar:/home/hadoop/hbase/lib/high-scale-lib-1.1.1.jar:/home/hadoop/hbase/lib/httpclient-4.0.1.jar:/home/hadoop/hbase/lib/httpcore-4.0.1.jar:/home/hadoop/hbase/lib/jackson-core-asl-1.5.5.jar:/home/hadoop/hbase/lib/jackson-jaxrs-1.5.5.jar:/home/hadoop/hbase/lib/jackson-mapper-asl-1.5.5.jar:/home/hadoop/hbase/lib/jackson-xc-1.5.5.jar:/home/hadoop/hbase/lib/jamon-runtime-2.3.1.jar:/home/hadoop/hbase/lib/jasper-compiler-5.5.23.jar:/home/hadoop/hbase/lib/jasper-runtime-5.5.23.jar:/home/hadoop/hbase/lib/jaxb-api-2.1.jar:/home/hadoop/hbase/lib/jaxb-impl-2.1.12.jar:/home/hadoop/hbase/lib/jersey-core-1.4.jar:/home/hadoop/hbase/lib/jersey-json-1.4.jar:/home/hadoop/hbase/lib/jersey-server-1.4.jar:/home/hadoop/hbase/lib/jettison-1.1.jar:/home/hadoop/hbase/lib/jetty-6.1.26.jar:/home/hadoop/hbase/lib/jetty-util-6.1.26.jar:/home/hadoop/hbase/lib/jruby-complete-1.6.5.jar:/home/hadoop/hbase/lib/jsp-2.1-6.1.14.jar:/home/hadoop/hbase/lib/jsp-api-2.1-6.1.14.jar:/home/hadoop/hbase/lib/libthrift-0.7.0.jar:/home/hadoop/hbase/lib/log4j-1.2.16.jar:/home/hadoop/hbase/lib/netty-3.2.4.Final.jar:/home/hadoop/hbase/lib/protobuf-java-2.4.0a.jar:/home/hadoop/hbase/lib/servlet-api-2.5-6.1.14.jar:/home/hadoop/hbase/lib/servlet-api-2.5.jar:/home/hadoop/hbase/lib/slf4j-api-1.5.8.jar:/home/hadoop/hbase/lib/slf4j-log4j12-1.5.8.jar:/home/hadoop/hbase/lib/snappy-java-1.0.3.2.jar:/home/hadoop/hbase/lib/stax-api-1.0.1.jar:/home/hadoop/hbase/lib/velocity-1.7.jar:/home/hadoop/hbase/lib/xmlenc-0.52.jar:/home/hadoop/hbase/lib/zookeeper-3.4.3.jar:
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:java.library.path=/home/hadoop/hbase/lib/native/Linux-amd64-64
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:os.version=2.6.35-30-generic
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:user.home=/home/hadoop/
> 2012-10-01 19:54:12,222 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Client
> environment:user.dir=/home/gregross
> 2012-10-01 19:54:12,225 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Initiating client connection,
> connectString=namenode-sn301.ngpipes.milp.ngmoco.com:2181
> sessionTimeout=180000 watcher=regionserver:60020
> 2012-10-01 19:54:12,251 [regionserver60020-SendThread()] INFO
> org.apache.zookeeper.ClientCnxn: Opening socket connection to server
> /10.100.102.197:2181
> 2012-10-01 19:54:12,252 [regionserver60020] INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier
> of this process is 15403@data3024.ngpipes.milp.ngmoco.com
> 2012-10-01 19:54:12,259
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-10-01 19:54:12,260
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181, initiating
> session
> 2012-10-01 19:54:12,272
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old
> server; r-o mode will be unavailable
> 2012-10-01 19:54:12,273
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
> on server namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181,
> sessionid = 0x137ec64373dd4b5, negotiated timeout = 40000
> 2012-10-01 19:54:12,289 [main] INFO
> org.apache.hadoop.hbase.regionserver.ShutdownHook: Installed shutdown
> hook thread: Shutdownhook:regionserver60020
> 2012-10-01 19:54:12,352 [regionserver60020] INFO
> org.apache.zookeeper.ZooKeeper: Initiating client connection,
> connectString=namenode-sn301.ngpipes.milp.ngmoco.com:2181
> sessionTimeout=180000 watcher=hconnection
> 2012-10-01 19:54:12,353 [regionserver60020-SendThread()] INFO
> org.apache.zookeeper.ClientCnxn: Opening socket connection to server
> /10.100.102.197:2181
> 2012-10-01 19:54:12,353 [regionserver60020] INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier
> of this process is 15403@data3024.ngpipes.milp.ngmoco.com
> 2012-10-01 19:54:12,354
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore
> this. On the other hand, if you expected SASL to work, please fix your
> JAAS configuration.
> 2012-10-01 19:54:12,354
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Socket connection established to
> namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181, initiating
> session
> 2012-10-01 19:54:12,361
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old
> server; r-o mode will be unavailable
> 2012-10-01 19:54:12,361
> [regionserver60020-SendThread(namenode-sn301.ngpipes.milp.ngmoco.com:2181)]
> INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
> on server namenode-sn301.ngpipes.milp.ngmoco.com/10.100.102.197:2181,
> sessionid = 0x137ec64373dd4b6, negotiated timeout = 40000
> 2012-10-01 19:54:12,384 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher:
> globalMemStoreLimit=1.6g, globalMemStoreLimitLowMark=1.4g,
> maxHeap=3.9g
> 2012-10-01 19:54:12,400 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Runs every 2hrs,
> 46mins, 40sec
> 2012-10-01 19:54:12,420 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting connect
> to Master server at
> namenode-sn301.ngpipes.milp.ngmoco.com,60000,1348698078915
> 2012-10-01 19:54:12,453 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to
> master at data3024.ngpipes.milp.ngmoco.com/10.100.101.156:60020
> 2012-10-01 19:54:12,453 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at
> namenode-sn301.ngpipes.milp.ngmoco.com,60000,1348698078915 that we are
> up with port=60020, startcode=1349121252040
> 2012-10-01 19:54:12,476 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us
> hostname to use. Was=data3024.ngpipes.milp.ngmoco.com,
> Now=data3024.ngpipes.milp.ngmoco.com
> 2012-10-01 19:54:12,568 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: HLog configuration:
> blocksize=64 MB, rollsize=60.8 MB, enabled=true,
> optionallogflushinternal=1000ms
> 2012-10-01 19:54:12,642 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog:  for
> /hbase/.logs/data3024.ngpipes.milp.ngmoco.com,60020,1349121252040/data3024.ngpipes.milp.ngmoco.com%2C60020%2C1349121252040.1349121252569
> 2012-10-01 19:54:12,643 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.wal.HLog: Using
> getNumCurrentReplicas--HDFS-826
> 2012-10-01 19:54:12,651 [regionserver60020] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
> with processName=RegionServer, sessionId=regionserver60020
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: revision
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUser
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: hdfsDate
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUrl
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: date
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: hdfsRevision
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: user
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: hdfsVersion
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: url
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: MetricsString added: version
> 2012-10-01 19:54:12,656 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: new MBeanInfo
> 2012-10-01 19:54:12,657 [regionserver60020] INFO
> org.apache.hadoop.hbase.metrics: new MBeanInfo
> 2012-10-01 19:54:12,657 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics:
> Initialized
> 2012-10-01 19:54:12,722 [regionserver60020] INFO org.mortbay.log:
> Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 2012-10-01 19:54:12,774 [regionserver60020] INFO
> org.apache.hadoop.http.HttpServer: Added global filtersafety
> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
> 2012-10-01 19:54:12,787 [regionserver60020] INFO
> org.apache.hadoop.http.HttpServer: Port returned by
> webServer.getConnectors()[0].getLocalPort() before open() is -1.
> Opening the listener on 60030
> 2012-10-01 19:54:12,787 [regionserver60020] INFO
> org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned
> 60030 webServer.getConnectors()[0].getLocalPort() returned 60030
> 2012-10-01 19:54:12,787 [regionserver60020] INFO
> org.apache.hadoop.http.HttpServer: Jetty bound to port 60030
> 2012-10-01 19:54:12,787 [regionserver60020] INFO org.mortbay.log: jetty-6.1.26
> 2012-10-01 19:54:13,079 [regionserver60020] INFO org.mortbay.log:
> Started SelectChannelConnector@0.0.0.0:60030
> 2012-10-01 19:54:13,079 [IPC Server Responder] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting
> 2012-10-01 19:54:13,079 [IPC Server listener on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60020:
> starting
> 2012-10-01 19:54:13,094 [IPC Server handler 0 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60020:
> starting
> 2012-10-01 19:54:13,094 [IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 2 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 3 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 4 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 5 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 6 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 7 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60020:
> starting
> 2012-10-01 19:54:13,095 [IPC Server handler 8 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 9 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 10 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 10 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 11 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 11 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 12 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 12 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 13 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 13 on 60020:
> starting
> 2012-10-01 19:54:13,096 [IPC Server handler 14 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 14 on 60020:
> starting
> 2012-10-01 19:54:13,097 [IPC Server handler 15 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 15 on 60020:
> starting
> 2012-10-01 19:54:13,097 [IPC Server handler 16 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 16 on 60020:
> starting
> 2012-10-01 19:54:13,097 [IPC Server handler 17 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 17 on 60020:
> starting
> 2012-10-01 19:54:13,097 [IPC Server handler 18 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 18 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 19 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 19 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 20 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 20 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 21 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 21 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 22 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 22 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 23 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 23 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 24 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 24 on 60020:
> starting
> 2012-10-01 19:54:13,098 [IPC Server handler 25 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 25 on 60020:
> starting
> 2012-10-01 19:54:13,099 [IPC Server handler 26 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 26 on 60020:
> starting
> 2012-10-01 19:54:13,099 [IPC Server handler 27 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 27 on 60020:
> starting
> 2012-10-01 19:54:13,099 [IPC Server handler 28 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 28 on 60020:
> starting
> 2012-10-01 19:54:13,100 [IPC Server handler 29 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 29 on 60020:
> starting
> 2012-10-01 19:54:13,101 [IPC Server handler 30 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 30 on 60020:
> starting
> 2012-10-01 19:54:13,101 [IPC Server handler 31 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 31 on 60020:
> starting
> 2012-10-01 19:54:13,101 [IPC Server handler 32 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 32 on 60020:
> starting
> 2012-10-01 19:54:13,101 [IPC Server handler 33 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 33 on 60020:
> starting
> 2012-10-01 19:54:13,101 [IPC Server handler 34 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 34 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 35 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 35 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 36 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 36 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 37 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 37 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 38 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 38 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 39 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 39 on 60020:
> starting
> 2012-10-01 19:54:13,102 [IPC Server handler 40 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 40 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 41 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 41 on 60020:
> starting

> 2012-10-01 19:54:13,103 [IPC Server handler 42 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 42 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 43 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 43 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 44 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 44 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 45 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 45 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 46 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 46 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 47 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 47 on 60020:
> starting
> 2012-10-01 19:54:13,103 [IPC Server handler 48 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 48 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 49 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 49 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 50 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 50 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 51 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 51 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 52 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 52 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 53 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 53 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 54 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 54 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 55 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 55 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 56 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 56 on 60020:
> starting
> 2012-10-01 19:54:13,104 [IPC Server handler 57 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 57 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 58 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 58 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 59 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 59 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 60 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 60 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 61 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 61 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 62 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 62 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 63 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 63 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 64 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 64 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 65 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 65 on 60020:
> starting
> 2012-10-01 19:54:13,105 [IPC Server handler 66 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 66 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 67 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 67 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 68 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 68 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 69 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 69 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 70 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 70 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 71 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 71 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 72 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 72 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 73 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 73 on 60020:
> starting
> 2012-10-01 19:54:13,106 [IPC Server handler 74 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 74 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 75 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 75 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 76 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 76 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 77 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 77 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 78 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 78 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 79 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 79 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 80 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 80 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 81 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 81 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 82 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 82 on 60020:
> starting
> 2012-10-01 19:54:13,107 [IPC Server handler 83 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 83 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 84 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 84 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 85 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 85 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 86 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 86 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 87 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 87 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 88 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 88 on 60020:
> starting
> 2012-10-01 19:54:13,108 [IPC Server handler 89 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 89 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 90 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 90 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 91 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 91 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 92 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 92 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 93 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 93 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 94 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 94 on 60020:
> starting
> 2012-10-01 19:54:13,109 [IPC Server handler 95 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 95 on 60020:
> starting
> 2012-10-01 19:54:13,110 [IPC Server handler 96 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 96 on 60020:
> starting
> 2012-10-01 19:54:13,110 [IPC Server handler 97 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 97 on 60020:
> starting
> 2012-10-01 19:54:13,110 [IPC Server handler 98 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 98 on 60020:
> starting
> 2012-10-01 19:54:13,110 [IPC Server handler 99 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: IPC Server handler 99 on 60020:
> starting
> 2012-10-01 19:54:13,110 [PRI IPC Server handler 0 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 0 on 60020:
> starting
> 2012-10-01 19:54:13,110 [PRI IPC Server handler 1 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 1 on 60020:
> starting
> 2012-10-01 19:54:13,110 [PRI IPC Server handler 2 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 2 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 3 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 3 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 4 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 4 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 5 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 5 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 6 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 6 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 7 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 7 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 8 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 8 on 60020:
> starting
> 2012-10-01 19:54:13,111 [PRI IPC Server handler 9 on 60020] INFO
> org.apache.hadoop.ipc.HBaseServer: PRI IPC Server handler 9 on 60020:
> starting
> 2012-10-01 19:54:13,124 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Serving as
> data3024.ngpipes.milp.ngmoco.com,60020,1349121252040, RPC listening on
> data3024.ngpipes.milp.ngmoco.com/10.100.101.156:60020,
> sessionid=0x137ec64373dd4b5
> 2012-10-01 19:54:13,124
> [SplitLogWorker-data3024.ngpipes.milp.ngmoco.com,60020,1349121252040]
> INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker:
> SplitLogWorker data3024.ngpipes.milp.ngmoco.com,60020,1349121252040
> starting
> 2012-10-01 19:54:13,125 [regionserver60020] INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Registered
> RegionServer MXBean
>
> GC log
> ======
>
> 1.914: [GC 1.914: [ParNew: 99976K->7646K(118016K), 0.0087130 secs]
> 99976K->7646K(123328K), 0.0088110 secs] [Times: user=0.07 sys=0.00,
> real=0.00 secs]
> 416.341: [GC 416.341: [ParNew: 112558K->12169K(118016K), 0.0447760
> secs] 112558K->25025K(133576K), 0.0450080 secs] [Times: user=0.13
> sys=0.02, real=0.05 secs]
> 416.386: [GC [1 CMS-initial-mark: 12855K(15560K)] 25089K(133576K),
> 0.0037570 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 416.390: [CMS-concurrent-mark-start]
> 416.407: [CMS-concurrent-mark: 0.015/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 416.407: [CMS-concurrent-preclean-start]
> 416.408: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 416.408: [GC[YG occupancy: 12233 K (118016 K)]416.408: [Rescan
> (parallel) , 0.0074970 secs]416.416: [weak refs processing, 0.0000370
> secs] [1 CMS-remark: 12855K(15560K)] 25089K(133576K), 0.0076480 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 416.416: [CMS-concurrent-sweep-start]
> 416.419: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 416.419: [CMS-concurrent-reset-start]
> 416.467: [CMS-concurrent-reset: 0.049/0.049 secs] [Times: user=0.01
> sys=0.04, real=0.05 secs]
> 418.468: [GC [1 CMS-initial-mark: 12855K(21428K)] 26216K(139444K),
> 0.0037020 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 418.471: [CMS-concurrent-mark-start]
> 418.487: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 418.487: [CMS-concurrent-preclean-start]
> 418.488: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 418.488: [GC[YG occupancy: 13360 K (118016 K)]418.488: [Rescan
> (parallel) , 0.0090770 secs]418.497: [weak refs processing, 0.0000170
> secs] [1 CMS-remark: 12855K(21428K)] 26216K(139444K), 0.0092220 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 418.497: [CMS-concurrent-sweep-start]
> 418.500: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 418.500: [CMS-concurrent-reset-start]
> 418.511: [CMS-concurrent-reset: 0.011/0.011 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 420.512: [GC [1 CMS-initial-mark: 12854K(21428K)] 26344K(139444K),
> 0.0041050 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 420.516: [CMS-concurrent-mark-start]
> 420.532: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.05
> sys=0.01, real=0.01 secs]
> 420.532: [CMS-concurrent-preclean-start]
> 420.533: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 420.533: [GC[YG occupancy: 13489 K (118016 K)]420.533: [Rescan
> (parallel) , 0.0014850 secs]420.534: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 12854K(21428K)] 26344K(139444K), 0.0015920 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 420.534: [CMS-concurrent-sweep-start]
> 420.537: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 420.537: [CMS-concurrent-reset-start]
> 420.548: [CMS-concurrent-reset: 0.011/0.011 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 422.437: [GC [1 CMS-initial-mark: 12854K(21428K)] 28692K(139444K),
> 0.0051030 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 422.443: [CMS-concurrent-mark-start]
> 422.458: [CMS-concurrent-mark: 0.014/0.015 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 422.458: [CMS-concurrent-preclean-start]
> 422.458: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 422.458: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 427.541:
> [CMS-concurrent-abortable-preclean: 0.678/5.083 secs] [Times:
> user=0.66 sys=0.00, real=5.08 secs]
> 427.541: [GC[YG occupancy: 16198 K (118016 K)]427.541: [Rescan
> (parallel) , 0.0013750 secs]427.543: [weak refs processing, 0.0000140
> secs] [1 CMS-remark: 12854K(21428K)] 29053K(139444K), 0.0014800 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 427.543: [CMS-concurrent-sweep-start]
> 427.544: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 427.544: [CMS-concurrent-reset-start]
> 427.557: [CMS-concurrent-reset: 0.013/0.013 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 429.557: [GC [1 CMS-initial-mark: 12854K(21428K)] 30590K(139444K),
> 0.0043280 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 429.562: [CMS-concurrent-mark-start]
> 429.574: [CMS-concurrent-mark: 0.012/0.012 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 429.574: [CMS-concurrent-preclean-start]
> 429.575: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 429.575: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 434.626:
> [CMS-concurrent-abortable-preclean: 0.747/5.051 secs] [Times:
> user=0.74 sys=0.00, real=5.05 secs]
> 434.626: [GC[YG occupancy: 18154 K (118016 K)]434.626: [Rescan
> (parallel) , 0.0015440 secs]434.627: [weak refs processing, 0.0000140
> secs] [1 CMS-remark: 12854K(21428K)] 31009K(139444K), 0.0016500 secs]
> [Times: user=0.00 sys=0.00, real=0.00 secs]
> 434.628: [CMS-concurrent-sweep-start]
> 434.629: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 434.629: [CMS-concurrent-reset-start]
> 434.641: [CMS-concurrent-reset: 0.012/0.012 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 436.641: [GC [1 CMS-initial-mark: 12854K(21428K)] 31137K(139444K),
> 0.0043440 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 436.646: [CMS-concurrent-mark-start]
> 436.660: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 436.660: [CMS-concurrent-preclean-start]
> 436.661: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 436.661: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 441.773:
> [CMS-concurrent-abortable-preclean: 0.608/5.112 secs] [Times:
> user=0.60 sys=0.00, real=5.11 secs]
> 441.773: [GC[YG occupancy: 18603 K (118016 K)]441.773: [Rescan
> (parallel) , 0.0024270 secs]441.776: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12854K(21428K)] 31458K(139444K), 0.0025200 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 441.776: [CMS-concurrent-sweep-start]
> 441.777: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 441.777: [CMS-concurrent-reset-start]
> 441.788: [CMS-concurrent-reset: 0.011/0.011 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 443.788: [GC [1 CMS-initial-mark: 12854K(21428K)] 31586K(139444K),
> 0.0044590 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 443.793: [CMS-concurrent-mark-start]
> 443.804: [CMS-concurrent-mark: 0.011/0.011 secs] [Times: user=0.04
> sys=0.00, real=0.02 secs]
> 443.804: [CMS-concurrent-preclean-start]
> 443.805: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 443.805: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 448.821:
> [CMS-concurrent-abortable-preclean: 0.813/5.016 secs] [Times:
> user=0.81 sys=0.00, real=5.01 secs]
> 448.822: [GC[YG occupancy: 19052 K (118016 K)]448.822: [Rescan
> (parallel) , 0.0013990 secs]448.823: [weak refs processing, 0.0000140
> secs] [1 CMS-remark: 12854K(21428K)] 31907K(139444K), 0.0015040 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 448.823: [CMS-concurrent-sweep-start]
> 448.825: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 448.825: [CMS-concurrent-reset-start]
> 448.837: [CMS-concurrent-reset: 0.012/0.012 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 450.837: [GC [1 CMS-initial-mark: 12854K(21428K)] 32035K(139444K),
> 0.0044510 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 450.842: [CMS-concurrent-mark-start]
> 450.857: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 450.857: [CMS-concurrent-preclean-start]
> 450.858: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 450.858: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 455.922:
> [CMS-concurrent-abortable-preclean: 0.726/5.064 secs] [Times:
> user=0.73 sys=0.00, real=5.06 secs]
> 455.922: [GC[YG occupancy: 19542 K (118016 K)]455.922: [Rescan
> (parallel) , 0.0016050 secs]455.924: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12854K(21428K)] 32397K(139444K), 0.0017340 secs]
> [Times: user=0.02 sys=0.00, real=0.01 secs]
> 455.924: [CMS-concurrent-sweep-start]
> 455.927: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 455.927: [CMS-concurrent-reset-start]
> 455.936: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 457.936: [GC [1 CMS-initial-mark: 12854K(21428K)] 32525K(139444K),
> 0.0026740 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 457.939: [CMS-concurrent-mark-start]
> 457.950: [CMS-concurrent-mark: 0.011/0.011 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 457.950: [CMS-concurrent-preclean-start]
> 457.950: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 457.950: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 463.065:
> [CMS-concurrent-abortable-preclean: 0.708/5.115 secs] [Times:
> user=0.71 sys=0.00, real=5.12 secs]
> 463.066: [GC[YG occupancy: 19991 K (118016 K)]463.066: [Rescan
> (parallel) , 0.0013940 secs]463.067: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12854K(21428K)] 32846K(139444K), 0.0015000 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 463.067: [CMS-concurrent-sweep-start]
> 463.070: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 463.070: [CMS-concurrent-reset-start]
> 463.080: [CMS-concurrent-reset: 0.010/0.010 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 465.080: [GC [1 CMS-initial-mark: 12854K(21428K)] 32974K(139444K),
> 0.0027070 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 465.083: [CMS-concurrent-mark-start]
> 465.096: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 465.096: [CMS-concurrent-preclean-start]
> 465.096: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 465.096: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 470.123:
> [CMS-concurrent-abortable-preclean: 0.723/5.027 secs] [Times:
> user=0.71 sys=0.00, real=5.03 secs]
> 470.124: [GC[YG occupancy: 20440 K (118016 K)]470.124: [Rescan
> (parallel) , 0.0011990 secs]470.125: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 12854K(21428K)] 33295K(139444K), 0.0012990 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 470.125: [CMS-concurrent-sweep-start]
> 470.127: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 470.127: [CMS-concurrent-reset-start]
> 470.137: [CMS-concurrent-reset: 0.010/0.010 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 472.137: [GC [1 CMS-initial-mark: 12854K(21428K)] 33423K(139444K),
> 0.0041330 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 472.141: [CMS-concurrent-mark-start]
> 472.155: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 472.155: [CMS-concurrent-preclean-start]
> 472.156: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 472.156: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 477.179:
> [CMS-concurrent-abortable-preclean: 0.618/5.023 secs] [Times:
> user=0.62 sys=0.00, real=5.02 secs]
> 477.179: [GC[YG occupancy: 20889 K (118016 K)]477.179: [Rescan
> (parallel) , 0.0014510 secs]477.180: [weak refs processing, 0.0000090
> secs] [1 CMS-remark: 12854K(21428K)] 33744K(139444K), 0.0015250 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 477.181: [CMS-concurrent-sweep-start]
> 477.183: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 477.183: [CMS-concurrent-reset-start]
> 477.192: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 479.192: [GC [1 CMS-initial-mark: 12854K(21428K)] 33872K(139444K),
> 0.0039730 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 479.196: [CMS-concurrent-mark-start]
> 479.209: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 479.209: [CMS-concurrent-preclean-start]
> 479.210: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 479.210: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 484.295:
> [CMS-concurrent-abortable-preclean: 0.757/5.085 secs] [Times:
> user=0.77 sys=0.00, real=5.09 secs]
> 484.295: [GC[YG occupancy: 21583 K (118016 K)]484.295: [Rescan
> (parallel) , 0.0013210 secs]484.297: [weak refs processing, 0.0000150
> secs] [1 CMS-remark: 12854K(21428K)] 34438K(139444K), 0.0014200 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 484.297: [CMS-concurrent-sweep-start]
> 484.298: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 484.298: [CMS-concurrent-reset-start]
> 484.307: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 486.308: [GC [1 CMS-initial-mark: 12854K(21428K)] 34566K(139444K),
> 0.0041800 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 486.312: [CMS-concurrent-mark-start]
> 486.324: [CMS-concurrent-mark: 0.012/0.012 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 486.324: [CMS-concurrent-preclean-start]
> 486.324: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 486.324: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 491.394:
> [CMS-concurrent-abortable-preclean: 0.565/5.070 secs] [Times:
> user=0.56 sys=0.00, real=5.06 secs]
> 491.394: [GC[YG occupancy: 22032 K (118016 K)]491.395: [Rescan
> (parallel) , 0.0018030 secs]491.396: [weak refs processing, 0.0000090
> secs] [1 CMS-remark: 12854K(21428K)] 34887K(139444K), 0.0018830 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 491.397: [CMS-concurrent-sweep-start]
> 491.398: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 491.398: [CMS-concurrent-reset-start]
> 491.406: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 493.407: [GC [1 CMS-initial-mark: 12854K(21428K)] 35080K(139444K),
> 0.0027620 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 493.410: [CMS-concurrent-mark-start]
> 493.420: [CMS-concurrent-mark: 0.010/0.010 secs] [Times: user=0.04
> sys=0.00, real=0.01 secs]
> 493.420: [CMS-concurrent-preclean-start]
> 493.420: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 493.420: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 498.525:
> [CMS-concurrent-abortable-preclean: 0.600/5.106 secs] [Times:
> user=0.61 sys=0.00, real=5.11 secs]
> 498.526: [GC[YG occupancy: 22545 K (118016 K)]498.526: [Rescan
> (parallel) , 0.0019450 secs]498.528: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12854K(21428K)] 35400K(139444K), 0.0020460 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 498.528: [CMS-concurrent-sweep-start]
> 498.530: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 498.530: [CMS-concurrent-reset-start]
> 498.538: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 500.538: [GC [1 CMS-initial-mark: 12854K(21428K)] 35529K(139444K),
> 0.0027790 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 500.541: [CMS-concurrent-mark-start]
> 500.554: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 500.554: [CMS-concurrent-preclean-start]
> 500.554: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 500.554: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 505.616:
> [CMS-concurrent-abortable-preclean: 0.557/5.062 secs] [Times:
> user=0.56 sys=0.00, real=5.06 secs]
> 505.617: [GC[YG occupancy: 22995 K (118016 K)]505.617: [Rescan
> (parallel) , 0.0023440 secs]505.619: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12854K(21428K)] 35850K(139444K), 0.0024280 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 505.619: [CMS-concurrent-sweep-start]
> 505.621: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 505.621: [CMS-concurrent-reset-start]
> 505.629: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 507.630: [GC [1 CMS-initial-mark: 12854K(21428K)] 35978K(139444K),
> 0.0027500 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 507.632: [CMS-concurrent-mark-start]
> 507.645: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 507.645: [CMS-concurrent-preclean-start]
> 507.646: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 507.646: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 512.697:
> [CMS-concurrent-abortable-preclean: 0.562/5.051 secs] [Times:
> user=0.57 sys=0.00, real=5.05 secs]
> 512.697: [GC[YG occupancy: 23484 K (118016 K)]512.697: [Rescan
> (parallel) , 0.0020030 secs]512.699: [weak refs processing, 0.0000090
> secs] [1 CMS-remark: 12854K(21428K)] 36339K(139444K), 0.0020830 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 512.700: [CMS-concurrent-sweep-start]
> 512.701: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 512.701: [CMS-concurrent-reset-start]
> 512.709: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 514.710: [GC [1 CMS-initial-mark: 12854K(21428K)] 36468K(139444K),
> 0.0028400 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 514.713: [CMS-concurrent-mark-start]
> 514.725: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 514.725: [CMS-concurrent-preclean-start]
> 514.725: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 514.725: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 519.800:
> [CMS-concurrent-abortable-preclean: 0.619/5.075 secs] [Times:
> user=0.66 sys=0.00, real=5.07 secs]
> 519.801: [GC[YG occupancy: 25022 K (118016 K)]519.801: [Rescan
> (parallel) , 0.0023950 secs]519.803: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12854K(21428K)] 37877K(139444K), 0.0024980 secs]
> [Times: user=0.02 sys=0.00, real=0.01 secs]
> 519.803: [CMS-concurrent-sweep-start]
> 519.805: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 519.805: [CMS-concurrent-reset-start]
> 519.813: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 521.814: [GC [1 CMS-initial-mark: 12854K(21428K)] 38005K(139444K),
> 0.0045520 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 521.818: [CMS-concurrent-mark-start]
> 521.833: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 521.833: [CMS-concurrent-preclean-start]
> 521.833: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 521.833: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 526.840:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 526.840: [GC[YG occupancy: 25471 K (118016 K)]526.840: [Rescan
> (parallel) , 0.0024440 secs]526.843: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12854K(21428K)] 38326K(139444K), 0.0025440 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 526.843: [CMS-concurrent-sweep-start]
> 526.845: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 526.845: [CMS-concurrent-reset-start]
> 526.853: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 528.853: [GC [1 CMS-initial-mark: 12849K(21428K)] 38449K(139444K),
> 0.0045550 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 528.858: [CMS-concurrent-mark-start]
> 528.872: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 528.872: [CMS-concurrent-preclean-start]
> 528.873: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 528.873: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 533.876:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 533.876: [GC[YG occupancy: 25919 K (118016 K)]533.877: [Rescan
> (parallel) , 0.0028370 secs]533.879: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 38769K(139444K), 0.0029390 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 533.880: [CMS-concurrent-sweep-start]
> 533.882: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 533.882: [CMS-concurrent-reset-start]
> 533.891: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 535.891: [GC [1 CMS-initial-mark: 12849K(21428K)] 38897K(139444K),
> 0.0046460 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 535.896: [CMS-concurrent-mark-start]
> 535.910: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 535.910: [CMS-concurrent-preclean-start]
> 535.911: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 535.911: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 540.917:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 540.917: [GC[YG occupancy: 26367 K (118016 K)]540.917: [Rescan
> (parallel) , 0.0025680 secs]540.920: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 39217K(139444K), 0.0026690 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 540.920: [CMS-concurrent-sweep-start]
> 540.922: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 540.922: [CMS-concurrent-reset-start]
> 540.930: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 542.466: [GC [1 CMS-initial-mark: 12849K(21428K)] 39555K(139444K),
> 0.0050040 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 542.471: [CMS-concurrent-mark-start]
> 542.486: [CMS-concurrent-mark: 0.014/0.015 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 542.486: [CMS-concurrent-preclean-start]
> 542.486: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 542.486: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 547.491:
> [CMS-concurrent-abortable-preclean: 0.699/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 547.491: [GC[YG occupancy: 27066 K (118016 K)]547.491: [Rescan
> (parallel) , 0.0024720 secs]547.494: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 39916K(139444K), 0.0025720 secs]
> [Times: user=0.02 sys=0.00, real=0.01 secs]
> 547.494: [CMS-concurrent-sweep-start]
> 547.496: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 547.496: [CMS-concurrent-reset-start]
> 547.505: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 549.506: [GC [1 CMS-initial-mark: 12849K(21428K)] 40044K(139444K),
> 0.0048760 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 549.511: [CMS-concurrent-mark-start]
> 549.524: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 549.524: [CMS-concurrent-preclean-start]
> 549.525: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 549.525: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 554.530:
> [CMS-concurrent-abortable-preclean: 0.700/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 554.530: [GC[YG occupancy: 27515 K (118016 K)]554.530: [Rescan
> (parallel) , 0.0025270 secs]554.533: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 40364K(139444K), 0.0026190 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 554.533: [CMS-concurrent-sweep-start]
> 554.534: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 554.534: [CMS-concurrent-reset-start]
> 554.542: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 556.543: [GC [1 CMS-initial-mark: 12849K(21428K)] 40493K(139444K),
> 0.0048950 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 556.548: [CMS-concurrent-mark-start]
> 556.562: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 556.562: [CMS-concurrent-preclean-start]
> 556.562: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 556.563: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 561.565:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 561.566: [GC[YG occupancy: 27963 K (118016 K)]561.566: [Rescan
> (parallel) , 0.0025900 secs]561.568: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 40813K(139444K), 0.0026910 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 561.569: [CMS-concurrent-sweep-start]
> 561.570: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 561.570: [CMS-concurrent-reset-start]
> 561.578: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 563.579: [GC [1 CMS-initial-mark: 12849K(21428K)] 40941K(139444K),
> 0.0049390 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 563.584: [CMS-concurrent-mark-start]
> 563.598: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 563.598: [CMS-concurrent-preclean-start]
> 563.598: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 563.598: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 568.693:
> [CMS-concurrent-abortable-preclean: 0.717/5.095 secs] [Times:
> user=0.71 sys=0.00, real=5.09 secs]
> 568.694: [GC[YG occupancy: 28411 K (118016 K)]568.694: [Rescan
> (parallel) , 0.0035750 secs]568.697: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 41261K(139444K), 0.0036740 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 568.698: [CMS-concurrent-sweep-start]
> 568.700: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 568.700: [CMS-concurrent-reset-start]
> 568.709: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 570.709: [GC [1 CMS-initial-mark: 12849K(21428K)] 41389K(139444K),
> 0.0048710 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 570.714: [CMS-concurrent-mark-start]
> 570.729: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 570.729: [CMS-concurrent-preclean-start]
> 570.729: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 570.729: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 575.738:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 575.738: [GC[YG occupancy: 28900 K (118016 K)]575.738: [Rescan
> (parallel) , 0.0036390 secs]575.742: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 41750K(139444K), 0.0037440 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 575.742: [CMS-concurrent-sweep-start]
> 575.744: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 575.744: [CMS-concurrent-reset-start]
> 575.752: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 577.752: [GC [1 CMS-initial-mark: 12849K(21428K)] 41878K(139444K),
> 0.0050100 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 577.758: [CMS-concurrent-mark-start]
> 577.772: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 577.772: [CMS-concurrent-preclean-start]
> 577.773: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 577.773: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 582.779:
> [CMS-concurrent-abortable-preclean: 0.701/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 582.779: [GC[YG occupancy: 29348 K (118016 K)]582.779: [Rescan
> (parallel) , 0.0026100 secs]582.782: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 42198K(139444K), 0.0027110 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 582.782: [CMS-concurrent-sweep-start]
> 582.784: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 582.784: [CMS-concurrent-reset-start]
> 582.792: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 584.792: [GC [1 CMS-initial-mark: 12849K(21428K)] 42326K(139444K),
> 0.0050510 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 584.798: [CMS-concurrent-mark-start]
> 584.812: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 584.812: [CMS-concurrent-preclean-start]
> 584.813: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 584.813: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 589.819:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 589.819: [GC[YG occupancy: 29797 K (118016 K)]589.819: [Rescan
> (parallel) , 0.0039510 secs]589.823: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 42647K(139444K), 0.0040460 secs]
> [Times: user=0.03 sys=0.00, real=0.01 secs]
> 589.824: [CMS-concurrent-sweep-start]
> 589.826: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 589.826: [CMS-concurrent-reset-start]
> 589.835: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 591.835: [GC [1 CMS-initial-mark: 12849K(21428K)] 42775K(139444K),
> 0.0050090 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 591.840: [CMS-concurrent-mark-start]
> 591.855: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 591.855: [CMS-concurrent-preclean-start]
> 591.855: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 591.855: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 596.857:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 596.857: [GC[YG occupancy: 31414 K (118016 K)]596.857: [Rescan
> (parallel) , 0.0028500 secs]596.860: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 44264K(139444K), 0.0029480 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 596.861: [CMS-concurrent-sweep-start]
> 596.862: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 596.862: [CMS-concurrent-reset-start]
> 596.870: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 598.871: [GC [1 CMS-initial-mark: 12849K(21428K)] 44392K(139444K),
> 0.0050640 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 598.876: [CMS-concurrent-mark-start]
> 598.890: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 598.890: [CMS-concurrent-preclean-start]
> 598.891: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 598.891: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 603.897:
> [CMS-concurrent-abortable-preclean: 0.705/5.007 secs] [Times:
> user=0.72 sys=0.00, real=5.01 secs]
> 603.898: [GC[YG occupancy: 32032 K (118016 K)]603.898: [Rescan
> (parallel) , 0.0039660 secs]603.902: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 44882K(139444K), 0.0040680 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 603.902: [CMS-concurrent-sweep-start]
> 603.903: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 603.903: [CMS-concurrent-reset-start]
> 603.912: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 605.912: [GC [1 CMS-initial-mark: 12849K(21428K)] 45010K(139444K),
> 0.0053650 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 605.918: [CMS-concurrent-mark-start]
> 605.932: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 605.932: [CMS-concurrent-preclean-start]
> 605.932: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 605.932: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 610.939:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 610.940: [GC[YG occupancy: 32481 K (118016 K)]610.940: [Rescan
> (parallel) , 0.0032540 secs]610.943: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 45330K(139444K), 0.0033560 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 610.943: [CMS-concurrent-sweep-start]
> 610.944: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 610.945: [CMS-concurrent-reset-start]
> 610.953: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 612.486: [GC [1 CMS-initial-mark: 12849K(21428K)] 45459K(139444K),
> 0.0055070 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 612.492: [CMS-concurrent-mark-start]
> 612.505: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 612.505: [CMS-concurrent-preclean-start]
> 612.506: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 612.506: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 617.511:
> [CMS-concurrent-abortable-preclean: 0.700/5.006 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 617.512: [GC[YG occupancy: 32929 K (118016 K)]617.512: [Rescan
> (parallel) , 0.0037500 secs]617.516: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 45779K(139444K), 0.0038560 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 617.516: [CMS-concurrent-sweep-start]
> 617.518: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 617.518: [CMS-concurrent-reset-start]
> 617.527: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 619.528: [GC [1 CMS-initial-mark: 12849K(21428K)] 45907K(139444K),
> 0.0053320 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 619.533: [CMS-concurrent-mark-start]
> 619.546: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 619.546: [CMS-concurrent-preclean-start]
> 619.547: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 619.547: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 624.552:
> [CMS-concurrent-abortable-preclean: 0.700/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 624.552: [GC[YG occupancy: 33377 K (118016 K)]624.552: [Rescan
> (parallel) , 0.0037290 secs]624.556: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 12849K(21428K)] 46227K(139444K), 0.0038330 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 624.556: [CMS-concurrent-sweep-start]
> 624.558: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 624.558: [CMS-concurrent-reset-start]
> 624.568: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 626.568: [GC [1 CMS-initial-mark: 12849K(21428K)] 46355K(139444K),
> 0.0054240 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 626.574: [CMS-concurrent-mark-start]
> 626.588: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 626.588: [CMS-concurrent-preclean-start]
> 626.588: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 626.588: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 631.592:
> [CMS-concurrent-abortable-preclean: 0.699/5.004 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 631.592: [GC[YG occupancy: 33825 K (118016 K)]631.593: [Rescan
> (parallel) , 0.0041600 secs]631.597: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 46675K(139444K), 0.0042650 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 631.597: [CMS-concurrent-sweep-start]
> 631.598: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 631.598: [CMS-concurrent-reset-start]
> 631.607: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 632.495: [GC [1 CMS-initial-mark: 12849K(21428K)] 46839K(139444K),
> 0.0054380 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 632.501: [CMS-concurrent-mark-start]
> 632.516: [CMS-concurrent-mark: 0.014/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 632.516: [CMS-concurrent-preclean-start]
> 632.517: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 632.517: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 637.519:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 637.519: [GC[YG occupancy: 34350 K (118016 K)]637.519: [Rescan
> (parallel) , 0.0025310 secs]637.522: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 47200K(139444K), 0.0026540 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 637.522: [CMS-concurrent-sweep-start]
> 637.523: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 637.523: [CMS-concurrent-reset-start]
> 637.532: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 639.532: [GC [1 CMS-initial-mark: 12849K(21428K)] 47328K(139444K),
> 0.0055330 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 639.538: [CMS-concurrent-mark-start]
> 639.551: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 639.551: [CMS-concurrent-preclean-start]
> 639.552: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 639.552: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 644.561:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 644.561: [GC[YG occupancy: 34798 K (118016 K)]644.561: [Rescan
> (parallel) , 0.0040620 secs]644.565: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 47648K(139444K), 0.0041610 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 644.566: [CMS-concurrent-sweep-start]
> 644.568: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 644.568: [CMS-concurrent-reset-start]
> 644.577: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 646.577: [GC [1 CMS-initial-mark: 12849K(21428K)] 47776K(139444K),
> 0.0054660 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 646.583: [CMS-concurrent-mark-start]
> 646.596: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 646.596: [CMS-concurrent-preclean-start]
> 646.597: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 646.597: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 651.678:
> [CMS-concurrent-abortable-preclean: 0.732/5.081 secs] [Times:
> user=0.74 sys=0.00, real=5.08 secs]
> 651.678: [GC[YG occupancy: 35246 K (118016 K)]651.678: [Rescan
> (parallel) , 0.0025920 secs]651.681: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 48096K(139444K), 0.0026910 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 651.681: [CMS-concurrent-sweep-start]
> 651.682: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 651.682: [CMS-concurrent-reset-start]
> 651.690: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 653.691: [GC [1 CMS-initial-mark: 12849K(21428K)] 48224K(139444K),
> 0.0055640 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 653.696: [CMS-concurrent-mark-start]
> 653.711: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 653.711: [CMS-concurrent-preclean-start]
> 653.711: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 653.711: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 658.721:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 658.721: [GC[YG occupancy: 35695 K (118016 K)]658.721: [Rescan
> (parallel) , 0.0040160 secs]658.725: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 48545K(139444K), 0.0041130 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 658.725: [CMS-concurrent-sweep-start]
> 658.727: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 658.728: [CMS-concurrent-reset-start]
> 658.737: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 660.737: [GC [1 CMS-initial-mark: 12849K(21428K)] 48673K(139444K),
> 0.0055230 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 660.743: [CMS-concurrent-mark-start]
> 660.756: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 660.756: [CMS-concurrent-preclean-start]
> 660.757: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 660.757: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 665.767:
> [CMS-concurrent-abortable-preclean: 0.704/5.011 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 665.768: [GC[YG occupancy: 36289 K (118016 K)]665.768: [Rescan
> (parallel) , 0.0033040 secs]665.771: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 49139K(139444K), 0.0034090 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 665.771: [CMS-concurrent-sweep-start]
> 665.773: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 665.773: [CMS-concurrent-reset-start]
> 665.781: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 667.781: [GC [1 CMS-initial-mark: 12849K(21428K)] 49267K(139444K),
> 0.0057830 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 667.787: [CMS-concurrent-mark-start]
> 667.802: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 667.802: [CMS-concurrent-preclean-start]
> 667.802: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 667.802: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 672.809:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 672.810: [GC[YG occupancy: 36737 K (118016 K)]672.810: [Rescan
> (parallel) , 0.0037010 secs]672.813: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 49587K(139444K), 0.0038010 secs]
> [Times: user=0.03 sys=0.00, real=0.01 secs]
> 672.814: [CMS-concurrent-sweep-start]
> 672.815: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 672.815: [CMS-concurrent-reset-start]
> 672.824: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 674.824: [GC [1 CMS-initial-mark: 12849K(21428K)] 49715K(139444K),
> 0.0058920 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
> 674.830: [CMS-concurrent-mark-start]
> 674.845: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 674.845: [CMS-concurrent-preclean-start]
> 674.845: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 674.845: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 679.849:
> [CMS-concurrent-abortable-preclean: 0.699/5.004 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 679.850: [GC[YG occupancy: 37185 K (118016 K)]679.850: [Rescan
> (parallel) , 0.0033420 secs]679.853: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 50035K(139444K), 0.0034440 secs]
> [Times: user=0.02 sys=0.00, real=0.01 secs]
> 679.853: [CMS-concurrent-sweep-start]
> 679.855: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 679.855: [CMS-concurrent-reset-start]
> 679.863: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 681.864: [GC [1 CMS-initial-mark: 12849K(21428K)] 50163K(139444K),
> 0.0058780 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 681.870: [CMS-concurrent-mark-start]
> 681.884: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 681.884: [CMS-concurrent-preclean-start]
> 681.884: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 681.884: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 686.890:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 686.891: [GC[YG occupancy: 37634 K (118016 K)]686.891: [Rescan
> (parallel) , 0.0044480 secs]686.895: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 50483K(139444K), 0.0045570 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 686.896: [CMS-concurrent-sweep-start]
> 686.897: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 686.897: [CMS-concurrent-reset-start]
> 686.905: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 688.905: [GC [1 CMS-initial-mark: 12849K(21428K)] 50612K(139444K),
> 0.0058940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 688.911: [CMS-concurrent-mark-start]
> 688.925: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 688.925: [CMS-concurrent-preclean-start]
> 688.925: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 688.926: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 694.041:
> [CMS-concurrent-abortable-preclean: 0.718/5.115 secs] [Times:
> user=0.72 sys=0.00, real=5.11 secs]
> 694.041: [GC[YG occupancy: 38122 K (118016 K)]694.041: [Rescan
> (parallel) , 0.0028640 secs]694.044: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 50972K(139444K), 0.0029660 secs]
> [Times: user=0.03 sys=0.00, real=0.01 secs]
> 694.044: [CMS-concurrent-sweep-start]
> 694.046: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 694.046: [CMS-concurrent-reset-start]
> 694.054: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 696.054: [GC [1 CMS-initial-mark: 12849K(21428K)] 51100K(139444K),
> 0.0060550 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 696.060: [CMS-concurrent-mark-start]
> 696.074: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 696.074: [CMS-concurrent-preclean-start]
> 696.075: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 696.075: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 701.078:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 701.079: [GC[YG occupancy: 38571 K (118016 K)]701.079: [Rescan
> (parallel) , 0.0064210 secs]701.085: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 51421K(139444K), 0.0065220 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 701.085: [CMS-concurrent-sweep-start]
> 701.087: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 701.088: [CMS-concurrent-reset-start]
> 701.097: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 703.097: [GC [1 CMS-initial-mark: 12849K(21428K)] 51549K(139444K),
> 0.0058470 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 703.103: [CMS-concurrent-mark-start]
> 703.116: [CMS-concurrent-mark: 0.013/0.013 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 703.116: [CMS-concurrent-preclean-start]
> 703.117: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 703.117: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 708.125:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 708.125: [GC[YG occupancy: 39054 K (118016 K)]708.125: [Rescan
> (parallel) , 0.0037190 secs]708.129: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 51904K(139444K), 0.0038220 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 708.129: [CMS-concurrent-sweep-start]
> 708.131: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 708.131: [CMS-concurrent-reset-start]
> 708.139: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 710.139: [GC [1 CMS-initial-mark: 12849K(21428K)] 52032K(139444K),
> 0.0059770 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 710.145: [CMS-concurrent-mark-start]
> 710.158: [CMS-concurrent-mark: 0.012/0.012 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 710.158: [CMS-concurrent-preclean-start]
> 710.158: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 710.158: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 715.169:
> [CMS-concurrent-abortable-preclean: 0.705/5.011 secs] [Times:
> user=0.69 sys=0.01, real=5.01 secs]
> 715.169: [GC[YG occupancy: 39503 K (118016 K)]715.169: [Rescan
> (parallel) , 0.0042370 secs]715.173: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 52353K(139444K), 0.0043410 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 715.174: [CMS-concurrent-sweep-start]
> 715.176: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 715.176: [CMS-concurrent-reset-start]
> 715.185: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 717.185: [GC [1 CMS-initial-mark: 12849K(21428K)] 52481K(139444K),
> 0.0060050 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 717.191: [CMS-concurrent-mark-start]
> 717.205: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 717.205: [CMS-concurrent-preclean-start]
> 717.206: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 717.206: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 722.209:
> [CMS-concurrent-abortable-preclean: 0.699/5.004 secs] [Times:
> user=0.71 sys=0.00, real=5.00 secs]
> 722.210: [GC[YG occupancy: 40161 K (118016 K)]722.210: [Rescan
> (parallel) , 0.0041630 secs]722.214: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 53011K(139444K), 0.0042630 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 722.214: [CMS-concurrent-sweep-start]
> 722.216: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 722.216: [CMS-concurrent-reset-start]
> 722.226: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 722.521: [GC [1 CMS-initial-mark: 12849K(21428K)] 53099K(139444K),
> 0.0062380 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 722.528: [CMS-concurrent-mark-start]
> 722.544: [CMS-concurrent-mark: 0.015/0.016 secs] [Times: user=0.05
> sys=0.01, real=0.02 secs]
> 722.544: [CMS-concurrent-preclean-start]
> 722.544: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 722.544: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 727.558:
> [CMS-concurrent-abortable-preclean: 0.709/5.014 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 727.558: [GC[YG occupancy: 40610 K (118016 K)]727.558: [Rescan
> (parallel) , 0.0041700 secs]727.563: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 53460K(139444K), 0.0042780 secs]
> [Times: user=0.05 sys=0.00, real=0.00 secs]
> 727.563: [CMS-concurrent-sweep-start]
> 727.564: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 727.564: [CMS-concurrent-reset-start]
> 727.573: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.02 secs]
> 729.574: [GC [1 CMS-initial-mark: 12849K(21428K)] 53588K(139444K),
> 0.0062700 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 729.580: [CMS-concurrent-mark-start]
> 729.595: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.02 secs]
> 729.595: [CMS-concurrent-preclean-start]
> 729.595: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 729.595: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 734.597:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 734.597: [GC[YG occupancy: 41058 K (118016 K)]734.597: [Rescan
> (parallel) , 0.0053870 secs]734.603: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 53908K(139444K), 0.0054870 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 734.603: [CMS-concurrent-sweep-start]
> 734.604: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 734.604: [CMS-concurrent-reset-start]
> 734.614: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 734.877: [GC [1 CMS-initial-mark: 12849K(21428K)] 53908K(139444K),
> 0.0067230 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 734.884: [CMS-concurrent-mark-start]
> 734.899: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 734.899: [CMS-concurrent-preclean-start]
> 734.899: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 734.899: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 739.905:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 739.906: [GC[YG occupancy: 41379 K (118016 K)]739.906: [Rescan
> (parallel) , 0.0050680 secs]739.911: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 54228K(139444K), 0.0051690 secs]
> [Times: user=0.05 sys=0.00, real=0.00 secs]
> 739.911: [CMS-concurrent-sweep-start]
> 739.912: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 739.912: [CMS-concurrent-reset-start]
> 739.921: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 741.922: [GC [1 CMS-initial-mark: 12849K(21428K)] 54356K(139444K),
> 0.0062880 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 741.928: [CMS-concurrent-mark-start]
> 741.942: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 741.942: [CMS-concurrent-preclean-start]
> 741.943: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 741.943: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 747.059:
> [CMS-concurrent-abortable-preclean: 0.711/5.117 secs] [Times:
> user=0.71 sys=0.00, real=5.12 secs]
> 747.060: [GC[YG occupancy: 41827 K (118016 K)]747.060: [Rescan
> (parallel) , 0.0051040 secs]747.065: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 54677K(139444K), 0.0052090 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 747.065: [CMS-concurrent-sweep-start]
> 747.067: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 747.067: [CMS-concurrent-reset-start]
> 747.075: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 749.075: [GC [1 CMS-initial-mark: 12849K(21428K)] 54805K(139444K),
> 0.0063470 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 749.082: [CMS-concurrent-mark-start]
> 749.095: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 749.095: [CMS-concurrent-preclean-start]
> 749.096: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 749.096: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 754.175:
> [CMS-concurrent-abortable-preclean: 0.718/5.079 secs] [Times:
> user=0.72 sys=0.00, real=5.08 secs]
> 754.175: [GC[YG occupancy: 42423 K (118016 K)]754.175: [Rescan
> (parallel) , 0.0051290 secs]754.180: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 55273K(139444K), 0.0052290 secs]
> [Times: user=0.05 sys=0.00, real=0.00 secs]
> 754.181: [CMS-concurrent-sweep-start]
> 754.182: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 754.182: [CMS-concurrent-reset-start]
> 754.191: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 756.191: [GC [1 CMS-initial-mark: 12849K(21428K)] 55401K(139444K),
> 0.0064020 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 756.198: [CMS-concurrent-mark-start]
> 756.212: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 756.212: [CMS-concurrent-preclean-start]
> 756.213: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 756.213: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 761.217:
> [CMS-concurrent-abortable-preclean: 0.699/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 761.218: [GC[YG occupancy: 42871 K (118016 K)]761.218: [Rescan
> (parallel) , 0.0052310 secs]761.223: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 55721K(139444K), 0.0053300 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 761.223: [CMS-concurrent-sweep-start]
> 761.225: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 761.225: [CMS-concurrent-reset-start]
> 761.234: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 763.234: [GC [1 CMS-initial-mark: 12849K(21428K)] 55849K(139444K),
> 0.0045400 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 763.239: [CMS-concurrent-mark-start]
> 763.253: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 763.253: [CMS-concurrent-preclean-start]
> 763.253: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 763.253: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 768.348:
> [CMS-concurrent-abortable-preclean: 0.690/5.095 secs] [Times:
> user=0.69 sys=0.00, real=5.10 secs]
> 768.349: [GC[YG occupancy: 43320 K (118016 K)]768.349: [Rescan
> (parallel) , 0.0045140 secs]768.353: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 56169K(139444K), 0.0046170 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 768.353: [CMS-concurrent-sweep-start]
> 768.356: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 768.356: [CMS-concurrent-reset-start]
> 768.365: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 770.365: [GC [1 CMS-initial-mark: 12849K(21428K)] 56298K(139444K),
> 0.0063950 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 770.372: [CMS-concurrent-mark-start]
> 770.388: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 770.388: [CMS-concurrent-preclean-start]
> 770.388: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 770.388: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 775.400:
> [CMS-concurrent-abortable-preclean: 0.707/5.012 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 775.401: [GC[YG occupancy: 43768 K (118016 K)]775.401: [Rescan
> (parallel) , 0.0043990 secs]775.405: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 56618K(139444K), 0.0045000 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 775.405: [CMS-concurrent-sweep-start]
> 775.407: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 775.407: [CMS-concurrent-reset-start]
> 775.417: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 777.417: [GC [1 CMS-initial-mark: 12849K(21428K)] 56746K(139444K),
> 0.0064580 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 777.423: [CMS-concurrent-mark-start]
> 777.438: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 777.438: [CMS-concurrent-preclean-start]
> 777.439: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 777.439: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 782.448:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 782.448: [GC[YG occupancy: 44321 K (118016 K)]782.448: [Rescan
> (parallel) , 0.0054760 secs]782.454: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 57171K(139444K), 0.0055780 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 782.454: [CMS-concurrent-sweep-start]
> 782.455: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 782.455: [CMS-concurrent-reset-start]
> 782.464: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 782.543: [GC [1 CMS-initial-mark: 12849K(21428K)] 57235K(139444K),
> 0.0066970 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 782.550: [CMS-concurrent-mark-start]
> 782.567: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 782.567: [CMS-concurrent-preclean-start]
> 782.568: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 782.568: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 787.574:
> [CMS-concurrent-abortable-preclean: 0.700/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 787.574: [GC[YG occupancy: 44746 K (118016 K)]787.574: [Rescan
> (parallel) , 0.0049170 secs]787.579: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 57596K(139444K), 0.0050210 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 787.579: [CMS-concurrent-sweep-start]
> 787.581: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 787.581: [CMS-concurrent-reset-start]
> 787.590: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 789.591: [GC [1 CMS-initial-mark: 12849K(21428K)] 57724K(139444K),
> 0.0066850 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 789.598: [CMS-concurrent-mark-start]
> 789.614: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 789.614: [CMS-concurrent-preclean-start]
> 789.615: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 789.615: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 794.626:
> [CMS-concurrent-abortable-preclean: 0.706/5.011 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 794.627: [GC[YG occupancy: 45195 K (118016 K)]794.627: [Rescan
> (parallel) , 0.0056520 secs]794.632: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 58044K(139444K), 0.0057510 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 794.632: [CMS-concurrent-sweep-start]
> 794.634: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 794.634: [CMS-concurrent-reset-start]
> 794.643: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 796.643: [GC [1 CMS-initial-mark: 12849K(21428K)] 58172K(139444K),
> 0.0067410 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 796.650: [CMS-concurrent-mark-start]
> 796.666: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 796.666: [CMS-concurrent-preclean-start]
> 796.667: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 796.667: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 801.670:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 801.670: [GC[YG occupancy: 45643 K (118016 K)]801.670: [Rescan
> (parallel) , 0.0043550 secs]801.675: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 58493K(139444K), 0.0044580 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 801.675: [CMS-concurrent-sweep-start]
> 801.677: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 801.677: [CMS-concurrent-reset-start]
> 801.686: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 803.686: [GC [1 CMS-initial-mark: 12849K(21428K)] 58621K(139444K),
> 0.0067250 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 803.693: [CMS-concurrent-mark-start]
> 803.708: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 803.708: [CMS-concurrent-preclean-start]
> 803.709: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 803.709: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 808.717:
> [CMS-concurrent-abortable-preclean: 0.702/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 808.717: [GC[YG occupancy: 46091 K (118016 K)]808.717: [Rescan
> (parallel) , 0.0034790 secs]808.720: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 58941K(139444K), 0.0035820 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 808.721: [CMS-concurrent-sweep-start]
> 808.722: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 808.722: [CMS-concurrent-reset-start]
> 808.730: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 810.731: [GC [1 CMS-initial-mark: 12849K(21428K)] 59069K(139444K),
> 0.0067580 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 810.738: [CMS-concurrent-mark-start]
> 810.755: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 810.755: [CMS-concurrent-preclean-start]
> 810.755: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 810.755: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 815.823:
> [CMS-concurrent-abortable-preclean: 0.715/5.068 secs] [Times:
> user=0.72 sys=0.00, real=5.06 secs]
> 815.824: [GC[YG occupancy: 46580 K (118016 K)]815.824: [Rescan
> (parallel) , 0.0048490 secs]815.829: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 59430K(139444K), 0.0049600 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 815.829: [CMS-concurrent-sweep-start]
> 815.831: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 815.831: [CMS-concurrent-reset-start]
> 815.840: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 817.840: [GC [1 CMS-initial-mark: 12849K(21428K)] 59558K(139444K),
> 0.0068880 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 817.847: [CMS-concurrent-mark-start]
> 817.864: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 817.864: [CMS-concurrent-preclean-start]
> 817.865: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 817.865: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 822.868:
> [CMS-concurrent-abortable-preclean: 0.697/5.003 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 822.868: [GC[YG occupancy: 47028 K (118016 K)]822.868: [Rescan
> (parallel) , 0.0061120 secs]822.874: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 59878K(139444K), 0.0062150 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 822.874: [CMS-concurrent-sweep-start]
> 822.876: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 822.876: [CMS-concurrent-reset-start]
> 822.885: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 824.885: [GC [1 CMS-initial-mark: 12849K(21428K)] 60006K(139444K),
> 0.0068610 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 824.892: [CMS-concurrent-mark-start]
> 824.908: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 824.908: [CMS-concurrent-preclean-start]
> 824.908: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 824.908: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 829.914:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 829.915: [GC[YG occupancy: 47477 K (118016 K)]829.915: [Rescan
> (parallel) , 0.0034890 secs]829.918: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 60327K(139444K), 0.0035930 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 829.918: [CMS-concurrent-sweep-start]
> 829.920: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 829.920: [CMS-concurrent-reset-start]
> 829.930: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 831.930: [GC [1 CMS-initial-mark: 12849K(21428K)] 60455K(139444K),
> 0.0069040 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 831.937: [CMS-concurrent-mark-start]
> 831.953: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 831.953: [CMS-concurrent-preclean-start]
> 831.954: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 831.954: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 836.957:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.71 sys=0.00, real=5.00 secs]
> 836.957: [GC[YG occupancy: 47925 K (118016 K)]836.957: [Rescan
> (parallel) , 0.0060440 secs]836.963: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 60775K(139444K), 0.0061520 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 836.964: [CMS-concurrent-sweep-start]
> 836.965: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 836.965: [CMS-concurrent-reset-start]
> 836.974: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 838.974: [GC [1 CMS-initial-mark: 12849K(21428K)] 60903K(139444K),
> 0.0069860 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 838.982: [CMS-concurrent-mark-start]
> 838.997: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 838.998: [CMS-concurrent-preclean-start]
> 838.998: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 838.998: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 844.091:
> [CMS-concurrent-abortable-preclean: 0.718/5.093 secs] [Times:
> user=0.72 sys=0.00, real=5.09 secs]
> 844.092: [GC[YG occupancy: 48731 K (118016 K)]844.092: [Rescan
> (parallel) , 0.0052610 secs]844.097: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 61581K(139444K), 0.0053620 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 844.097: [CMS-concurrent-sweep-start]
> 844.099: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 844.099: [CMS-concurrent-reset-start]
> 844.108: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 846.109: [GC [1 CMS-initial-mark: 12849K(21428K)] 61709K(139444K),
> 0.0071980 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 846.116: [CMS-concurrent-mark-start]
> 846.133: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 846.133: [CMS-concurrent-preclean-start]
> 846.134: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 846.134: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 851.137:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 851.137: [GC[YG occupancy: 49180 K (118016 K)]851.137: [Rescan
> (parallel) , 0.0061320 secs]851.143: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 62030K(139444K), 0.0062320 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 851.144: [CMS-concurrent-sweep-start]
> 851.145: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 851.145: [CMS-concurrent-reset-start]
> 851.154: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 853.154: [GC [1 CMS-initial-mark: 12849K(21428K)] 62158K(139444K),
> 0.0071610 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 853.162: [CMS-concurrent-mark-start]
> 853.177: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 853.177: [CMS-concurrent-preclean-start]
> 853.178: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 853.178: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 858.181:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 858.181: [GC[YG occupancy: 49628 K (118016 K)]858.181: [Rescan
> (parallel) , 0.0029560 secs]858.184: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 62478K(139444K), 0.0030590 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 858.184: [CMS-concurrent-sweep-start]
> 858.186: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 858.186: [CMS-concurrent-reset-start]
> 858.195: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 860.196: [GC [1 CMS-initial-mark: 12849K(21428K)] 62606K(139444K),
> 0.0072070 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 860.203: [CMS-concurrent-mark-start]
> 860.219: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 860.219: [CMS-concurrent-preclean-start]
> 860.219: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 860.219: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 865.226:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 865.227: [GC[YG occupancy: 50076 K (118016 K)]865.227: [Rescan
> (parallel) , 0.0066610 secs]865.233: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 62926K(139444K), 0.0067670 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 865.233: [CMS-concurrent-sweep-start]
> 865.235: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 865.235: [CMS-concurrent-reset-start]
> 865.244: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 867.244: [GC [1 CMS-initial-mark: 12849K(21428K)] 63054K(139444K),
> 0.0072490 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 867.252: [CMS-concurrent-mark-start]
> 867.267: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 867.267: [CMS-concurrent-preclean-start]
> 867.268: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 867.268: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 872.281:
> [CMS-concurrent-abortable-preclean: 0.708/5.013 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 872.281: [GC[YG occupancy: 50525 K (118016 K)]872.281: [Rescan
> (parallel) , 0.0053780 secs]872.286: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 63375K(139444K), 0.0054790 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 872.287: [CMS-concurrent-sweep-start]
> 872.288: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 872.288: [CMS-concurrent-reset-start]
> 872.296: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 872.572: [GC [1 CMS-initial-mark: 12849K(21428K)] 63439K(139444K),
> 0.0073060 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 872.580: [CMS-concurrent-mark-start]
> 872.597: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 872.597: [CMS-concurrent-preclean-start]
> 872.597: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 872.597: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 877.600:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 877.601: [GC[YG occupancy: 51049 K (118016 K)]877.601: [Rescan
> (parallel) , 0.0063070 secs]877.607: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 63899K(139444K), 0.0064090 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 877.607: [CMS-concurrent-sweep-start]
> 877.609: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 877.609: [CMS-concurrent-reset-start]
> 877.619: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 879.619: [GC [1 CMS-initial-mark: 12849K(21428K)] 64027K(139444K),
> 0.0073320 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 879.626: [CMS-concurrent-mark-start]
> 879.643: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 879.643: [CMS-concurrent-preclean-start]
> 879.644: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 879.644: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 884.657:
> [CMS-concurrent-abortable-preclean: 0.708/5.014 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 884.658: [GC[YG occupancy: 51497 K (118016 K)]884.658: [Rescan
> (parallel) , 0.0056160 secs]884.663: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 64347K(139444K), 0.0057150 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 884.663: [CMS-concurrent-sweep-start]
> 884.665: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 884.665: [CMS-concurrent-reset-start]
> 884.674: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 886.674: [GC [1 CMS-initial-mark: 12849K(21428K)] 64475K(139444K),
> 0.0073420 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 886.682: [CMS-concurrent-mark-start]
> 886.698: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 886.698: [CMS-concurrent-preclean-start]
> 886.698: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 886.698: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 891.702:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 891.702: [GC[YG occupancy: 51945 K (118016 K)]891.702: [Rescan
> (parallel) , 0.0070120 secs]891.709: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 64795K(139444K), 0.0071150 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 891.709: [CMS-concurrent-sweep-start]
> 891.711: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 891.711: [CMS-concurrent-reset-start]
> 891.721: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 893.721: [GC [1 CMS-initial-mark: 12849K(21428K)] 64923K(139444K),
> 0.0073880 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 893.728: [CMS-concurrent-mark-start]
> 893.745: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 893.745: [CMS-concurrent-preclean-start]
> 893.745: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 893.745: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 898.852:
> [CMS-concurrent-abortable-preclean: 0.715/5.107 secs] [Times:
> user=0.71 sys=0.00, real=5.10 secs]
> 898.853: [GC[YG occupancy: 53466 K (118016 K)]898.853: [Rescan
> (parallel) , 0.0060600 secs]898.859: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 66315K(139444K), 0.0061640 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 898.859: [CMS-concurrent-sweep-start]
> 898.861: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 898.861: [CMS-concurrent-reset-start]
> 898.870: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 900.871: [GC [1 CMS-initial-mark: 12849K(21428K)] 66444K(139444K),
> 0.0074670 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 900.878: [CMS-concurrent-mark-start]
> 900.895: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 900.895: [CMS-concurrent-preclean-start]
> 900.896: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 900.896: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 905.969:
> [CMS-concurrent-abortable-preclean: 0.716/5.074 secs] [Times:
> user=0.72 sys=0.01, real=5.07 secs]
> 905.969: [GC[YG occupancy: 54157 K (118016 K)]905.970: [Rescan
> (parallel) , 0.0068200 secs]905.976: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 67007K(139444K), 0.0069250 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 905.977: [CMS-concurrent-sweep-start]
> 905.978: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 905.978: [CMS-concurrent-reset-start]
> 905.986: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 907.986: [GC [1 CMS-initial-mark: 12849K(21428K)] 67135K(139444K),
> 0.0076010 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 907.994: [CMS-concurrent-mark-start]
> 908.009: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 908.009: [CMS-concurrent-preclean-start]
> 908.010: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 908.010: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 913.013:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 913.013: [GC[YG occupancy: 54606 K (118016 K)]913.013: [Rescan
> (parallel) , 0.0053650 secs]913.018: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 67455K(139444K), 0.0054650 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 913.019: [CMS-concurrent-sweep-start]
> 913.021: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 913.021: [CMS-concurrent-reset-start]
> 913.030: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 915.030: [GC [1 CMS-initial-mark: 12849K(21428K)] 67583K(139444K),
> 0.0076410 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 915.038: [CMS-concurrent-mark-start]
> 915.055: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 915.055: [CMS-concurrent-preclean-start]
> 915.056: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 915.056: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 920.058:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 920.058: [GC[YG occupancy: 55054 K (118016 K)]920.058: [Rescan
> (parallel) , 0.0058380 secs]920.064: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 67904K(139444K), 0.0059420 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 920.064: [CMS-concurrent-sweep-start]
> 920.066: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 920.066: [CMS-concurrent-reset-start]
> 920.075: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.01, real=0.01 secs]
> 922.075: [GC [1 CMS-initial-mark: 12849K(21428K)] 68032K(139444K),
> 0.0075820 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 922.083: [CMS-concurrent-mark-start]
> 922.098: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 922.098: [CMS-concurrent-preclean-start]
> 922.099: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 922.099: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 927.102:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 927.102: [GC[YG occupancy: 55502 K (118016 K)]927.102: [Rescan
> (parallel) , 0.0059190 secs]927.108: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 68352K(139444K), 0.0060220 secs]
> [Times: user=0.06 sys=0.01, real=0.01 secs]
> 927.108: [CMS-concurrent-sweep-start]
> 927.110: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 927.110: [CMS-concurrent-reset-start]
> 927.120: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 929.120: [GC [1 CMS-initial-mark: 12849K(21428K)] 68480K(139444K),
> 0.0077620 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 929.128: [CMS-concurrent-mark-start]
> 929.145: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 929.145: [CMS-concurrent-preclean-start]
> 929.145: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 929.145: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 934.237:
> [CMS-concurrent-abortable-preclean: 0.717/5.092 secs] [Times:
> user=0.72 sys=0.00, real=5.09 secs]
> 934.238: [GC[YG occupancy: 55991 K (118016 K)]934.238: [Rescan
> (parallel) , 0.0042660 secs]934.242: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 68841K(139444K), 0.0043660 secs]
> [Times: user=0.05 sys=0.00, real=0.00 secs]
> 934.242: [CMS-concurrent-sweep-start]
> 934.244: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 934.244: [CMS-concurrent-reset-start]
> 934.252: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 936.253: [GC [1 CMS-initial-mark: 12849K(21428K)] 68969K(139444K),
> 0.0077340 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 936.261: [CMS-concurrent-mark-start]
> 936.277: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 936.277: [CMS-concurrent-preclean-start]
> 936.278: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 936.278: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 941.284:
> [CMS-concurrent-abortable-preclean: 0.701/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 941.284: [GC[YG occupancy: 56439 K (118016 K)]941.284: [Rescan
> (parallel) , 0.0059460 secs]941.290: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 69289K(139444K), 0.0060470 secs]
> [Times: user=0.08 sys=0.00, real=0.00 secs]
> 941.290: [CMS-concurrent-sweep-start]
> 941.293: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 941.293: [CMS-concurrent-reset-start]
> 941.302: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 943.302: [GC [1 CMS-initial-mark: 12849K(21428K)] 69417K(139444K),
> 0.0077760 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 943.310: [CMS-concurrent-mark-start]
> 943.326: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 943.326: [CMS-concurrent-preclean-start]
> 943.327: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 943.327: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 948.340:
> [CMS-concurrent-abortable-preclean: 0.708/5.013 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 948.340: [GC[YG occupancy: 56888 K (118016 K)]948.340: [Rescan
> (parallel) , 0.0047760 secs]948.345: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 69738K(139444K), 0.0048770 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 948.345: [CMS-concurrent-sweep-start]
> 948.347: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 948.347: [CMS-concurrent-reset-start]
> 948.356: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 950.356: [GC [1 CMS-initial-mark: 12849K(21428K)] 69866K(139444K),
> 0.0077750 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 950.364: [CMS-concurrent-mark-start]
> 950.380: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 950.380: [CMS-concurrent-preclean-start]
> 950.380: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 950.380: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 955.384:
> [CMS-concurrent-abortable-preclean: 0.698/5.004 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 955.384: [GC[YG occupancy: 57336 K (118016 K)]955.384: [Rescan
> (parallel) , 0.0072540 secs]955.392: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 70186K(139444K), 0.0073540 secs]
> [Times: user=0.08 sys=0.00, real=0.00 secs]
> 955.392: [CMS-concurrent-sweep-start]
> 955.394: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 955.394: [CMS-concurrent-reset-start]
> 955.403: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 957.403: [GC [1 CMS-initial-mark: 12849K(21428K)] 70314K(139444K),
> 0.0078120 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 957.411: [CMS-concurrent-mark-start]
> 957.427: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 957.427: [CMS-concurrent-preclean-start]
> 957.427: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 957.427: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 962.437:
> [CMS-concurrent-abortable-preclean: 0.704/5.010 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 962.437: [GC[YG occupancy: 57889 K (118016 K)]962.437: [Rescan
> (parallel) , 0.0076140 secs]962.445: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 70739K(139444K), 0.0077160 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 962.445: [CMS-concurrent-sweep-start]
> 962.446: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 962.446: [CMS-concurrent-reset-start]
> 962.456: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 962.599: [GC [1 CMS-initial-mark: 12849K(21428K)] 70827K(139444K),
> 0.0081180 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 962.608: [CMS-concurrent-mark-start]
> 962.626: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 962.626: [CMS-concurrent-preclean-start]
> 962.626: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 962.626: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 967.632:
> [CMS-concurrent-abortable-preclean: 0.700/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 967.632: [GC[YG occupancy: 58338 K (118016 K)]967.632: [Rescan
> (parallel) , 0.0061170 secs]967.638: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 71188K(139444K), 0.0062190 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 967.638: [CMS-concurrent-sweep-start]
> 967.640: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 967.640: [CMS-concurrent-reset-start]
> 967.648: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 969.648: [GC [1 CMS-initial-mark: 12849K(21428K)] 71316K(139444K),
> 0.0081110 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 969.656: [CMS-concurrent-mark-start]
> 969.674: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 969.674: [CMS-concurrent-preclean-start]
> 969.674: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 969.674: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 974.677:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:

> user=0.70 sys=0.00, real=5.00 secs]
> 974.677: [GC[YG occupancy: 58786 K (118016 K)]974.677: [Rescan
> (parallel) , 0.0070810 secs]974.685: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 71636K(139444K), 0.0072050 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 974.685: [CMS-concurrent-sweep-start]
> 974.686: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 974.686: [CMS-concurrent-reset-start]
> 974.695: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 976.696: [GC [1 CMS-initial-mark: 12849K(21428K)] 71764K(139444K),
> 0.0080650 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 976.704: [CMS-concurrent-mark-start]
> 976.719: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 976.719: [CMS-concurrent-preclean-start]
> 976.719: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 976.719: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 981.727:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.69 sys=0.01, real=5.01 secs]
> 981.727: [GC[YG occupancy: 59235 K (118016 K)]981.727: [Rescan
> (parallel) , 0.0066570 secs]981.734: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 72085K(139444K), 0.0067620 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 981.734: [CMS-concurrent-sweep-start]
> 981.736: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 981.736: [CMS-concurrent-reset-start]
> 981.745: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 983.745: [GC [1 CMS-initial-mark: 12849K(21428K)] 72213K(139444K),
> 0.0081400 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 983.753: [CMS-concurrent-mark-start]
> 983.769: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 983.769: [CMS-concurrent-preclean-start]
> 983.769: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 983.769: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 988.840:
> [CMS-concurrent-abortable-preclean: 0.716/5.071 secs] [Times:
> user=0.71 sys=0.00, real=5.07 secs]
> 988.840: [GC[YG occupancy: 59683 K (118016 K)]988.840: [Rescan
> (parallel) , 0.0076020 secs]988.848: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 72533K(139444K), 0.0077100 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 988.848: [CMS-concurrent-sweep-start]
> 988.850: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 988.850: [CMS-concurrent-reset-start]
> 988.858: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 990.858: [GC [1 CMS-initial-mark: 12849K(21428K)] 72661K(139444K),
> 0.0081810 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 990.867: [CMS-concurrent-mark-start]
> 990.884: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 990.884: [CMS-concurrent-preclean-start]
> 990.885: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 990.885: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 995.999:
> [CMS-concurrent-abortable-preclean: 0.721/5.114 secs] [Times:
> user=0.73 sys=0.00, real=5.11 secs]
> 995.999: [GC[YG occupancy: 60307 K (118016 K)]995.999: [Rescan
> (parallel) , 0.0058190 secs]996.005: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 73156K(139444K), 0.0059260 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 996.005: [CMS-concurrent-sweep-start]
> 996.007: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 996.007: [CMS-concurrent-reset-start]
> 996.016: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 998.016: [GC [1 CMS-initial-mark: 12849K(21428K)] 73285K(139444K),
> 0.0052760 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 998.022: [CMS-concurrent-mark-start]
> 998.038: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 998.038: [CMS-concurrent-preclean-start]
> 998.039: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 998.039: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1003.048:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1003.048: [GC[YG occupancy: 60755 K (118016 K)]1003.048: [Rescan
> (parallel) , 0.0068040 secs]1003.055: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 73605K(139444K), 0.0069060 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 1003.055: [CMS-concurrent-sweep-start]
> 1003.057: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1003.057: [CMS-concurrent-reset-start]
> 1003.066: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1005.066: [GC [1 CMS-initial-mark: 12849K(21428K)] 73733K(139444K),
> 0.0082200 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1005.075: [CMS-concurrent-mark-start]
> 1005.090: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1005.090: [CMS-concurrent-preclean-start]
> 1005.090: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1005.090: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1010.094:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1010.094: [GC[YG occupancy: 61203 K (118016 K)]1010.094: [Rescan
> (parallel) , 0.0066010 secs]1010.101: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 74053K(139444K), 0.0067120 secs]
> [Times: user=0.08 sys=0.00, real=0.00 secs]
> 1010.101: [CMS-concurrent-sweep-start]
> 1010.103: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1010.103: [CMS-concurrent-reset-start]
> 1010.112: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1012.113: [GC [1 CMS-initial-mark: 12849K(21428K)] 74181K(139444K),
> 0.0083460 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1012.121: [CMS-concurrent-mark-start]
> 1012.137: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1012.137: [CMS-concurrent-preclean-start]
> 1012.138: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1012.138: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1017.144:
> [CMS-concurrent-abortable-preclean: 0.701/5.007 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1017.144: [GC[YG occupancy: 61651 K (118016 K)]1017.144: [Rescan
> (parallel) , 0.0058810 secs]1017.150: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 74501K(139444K), 0.0059830 secs]
> [Times: user=0.06 sys=0.00, real=0.00 secs]
> 1017.151: [CMS-concurrent-sweep-start]
> 1017.153: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1017.153: [CMS-concurrent-reset-start]
> 1017.162: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1019.162: [GC [1 CMS-initial-mark: 12849K(21428K)] 74629K(139444K),
> 0.0083310 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1019.171: [CMS-concurrent-mark-start]
> 1019.187: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1019.187: [CMS-concurrent-preclean-start]
> 1019.187: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1019.187: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1024.261:
> [CMS-concurrent-abortable-preclean: 0.717/5.074 secs] [Times:
> user=0.72 sys=0.00, real=5.07 secs]
> 1024.261: [GC[YG occupancy: 62351 K (118016 K)]1024.262: [Rescan
> (parallel) , 0.0069720 secs]1024.269: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 75200K(139444K), 0.0070750 secs]
> [Times: user=0.08 sys=0.01, real=0.01 secs]
> 1024.269: [CMS-concurrent-sweep-start]
> 1024.270: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1024.270: [CMS-concurrent-reset-start]
> 1024.278: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1026.279: [GC [1 CMS-initial-mark: 12849K(21428K)] 75329K(139444K),
> 0.0086360 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1026.288: [CMS-concurrent-mark-start]
> 1026.305: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1026.305: [CMS-concurrent-preclean-start]
> 1026.305: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1026.305: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1031.308:
> [CMS-concurrent-abortable-preclean: 0.697/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1031.308: [GC[YG occupancy: 62799 K (118016 K)]1031.308: [Rescan
> (parallel) , 0.0069330 secs]1031.315: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 75649K(139444K), 0.0070380 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1031.315: [CMS-concurrent-sweep-start]
> 1031.316: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1031.316: [CMS-concurrent-reset-start]
> 1031.326: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1033.326: [GC [1 CMS-initial-mark: 12849K(21428K)] 75777K(139444K),
> 0.0085850 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1033.335: [CMS-concurrent-mark-start]
> 1033.350: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1033.350: [CMS-concurrent-preclean-start]
> 1033.351: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1033.351: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1038.357:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.69 sys=0.01, real=5.01 secs]
> 1038.358: [GC[YG occupancy: 63247 K (118016 K)]1038.358: [Rescan
> (parallel) , 0.0071860 secs]1038.365: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 76097K(139444K), 0.0072900 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 1038.365: [CMS-concurrent-sweep-start]
> 1038.367: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1038.367: [CMS-concurrent-reset-start]
> 1038.376: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1040.376: [GC [1 CMS-initial-mark: 12849K(21428K)] 76225K(139444K),
> 0.0085910 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1040.385: [CMS-concurrent-mark-start]
> 1040.401: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1040.401: [CMS-concurrent-preclean-start]
> 1040.401: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1040.401: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1045.411:
> [CMS-concurrent-abortable-preclean: 0.705/5.010 secs] [Times:
> user=0.69 sys=0.01, real=5.01 secs]
> 1045.412: [GC[YG occupancy: 63695 K (118016 K)]1045.412: [Rescan
> (parallel) , 0.0082050 secs]1045.420: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 76545K(139444K), 0.0083110 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1045.420: [CMS-concurrent-sweep-start]
> 1045.421: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1045.421: [CMS-concurrent-reset-start]
> 1045.430: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1047.430: [GC [1 CMS-initial-mark: 12849K(21428K)] 76673K(139444K),
> 0.0086110 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1047.439: [CMS-concurrent-mark-start]
> 1047.456: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1047.456: [CMS-concurrent-preclean-start]
> 1047.456: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1047.456: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1052.462:
> [CMS-concurrent-abortable-preclean: 0.700/5.005 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1052.462: [GC[YG occupancy: 64144 K (118016 K)]1052.462: [Rescan
> (parallel) , 0.0087770 secs]1052.471: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 76994K(139444K), 0.0088770 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1052.471: [CMS-concurrent-sweep-start]
> 1052.472: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1052.472: [CMS-concurrent-reset-start]
> 1052.481: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1052.628: [GC [1 CMS-initial-mark: 12849K(21428K)] 77058K(139444K),
> 0.0086170 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1052.637: [CMS-concurrent-mark-start]
> 1052.655: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1052.655: [CMS-concurrent-preclean-start]
> 1052.656: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1052.656: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1057.658:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1057.658: [GC[YG occupancy: 64569 K (118016 K)]1057.658: [Rescan
> (parallel) , 0.0072850 secs]1057.665: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 77418K(139444K), 0.0073880 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1057.666: [CMS-concurrent-sweep-start]
> 1057.668: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1057.668: [CMS-concurrent-reset-start]
> 1057.677: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1059.677: [GC [1 CMS-initial-mark: 12849K(21428K)] 77547K(139444K),
> 0.0086820 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1059.686: [CMS-concurrent-mark-start]
> 1059.703: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1059.703: [CMS-concurrent-preclean-start]
> 1059.703: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1059.703: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1064.712:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1064.712: [GC[YG occupancy: 65017 K (118016 K)]1064.712: [Rescan
> (parallel) , 0.0071630 secs]1064.720: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 77867K(139444K), 0.0072700 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1064.720: [CMS-concurrent-sweep-start]
> 1064.722: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1064.722: [CMS-concurrent-reset-start]
> 1064.731: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1066.731: [GC [1 CMS-initial-mark: 12849K(21428K)] 77995K(139444K),
> 0.0087640 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1066.740: [CMS-concurrent-mark-start]
> 1066.757: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1066.757: [CMS-concurrent-preclean-start]
> 1066.757: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1066.757: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1071.821:
> [CMS-concurrent-abortable-preclean: 0.714/5.064 secs] [Times:
> user=0.71 sys=0.00, real=5.06 secs]
> 1071.822: [GC[YG occupancy: 65465 K (118016 K)]1071.822: [Rescan
> (parallel) , 0.0056280 secs]1071.827: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 78315K(139444K), 0.0057430 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 1071.828: [CMS-concurrent-sweep-start]
> 1071.830: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1071.830: [CMS-concurrent-reset-start]
> 1071.839: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1073.839: [GC [1 CMS-initial-mark: 12849K(21428K)] 78443K(139444K),
> 0.0087570 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1073.848: [CMS-concurrent-mark-start]
> 1073.865: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1073.865: [CMS-concurrent-preclean-start]
> 1073.865: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1073.865: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1078.868:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1078.868: [GC[YG occupancy: 65914 K (118016 K)]1078.868: [Rescan
> (parallel) , 0.0055280 secs]1078.873: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 78763K(139444K), 0.0056320 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 1078.874: [CMS-concurrent-sweep-start]
> 1078.875: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1078.875: [CMS-concurrent-reset-start]
> 1078.884: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1080.884: [GC [1 CMS-initial-mark: 12849K(21428K)] 78892K(139444K),
> 0.0088520 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1080.893: [CMS-concurrent-mark-start]
> 1080.908: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1080.909: [CMS-concurrent-preclean-start]
> 1080.909: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1080.909: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1086.021:
> [CMS-concurrent-abortable-preclean: 0.714/5.112 secs] [Times:
> user=0.72 sys=0.00, real=5.11 secs]
> 1086.021: [GC[YG occupancy: 66531 K (118016 K)]1086.022: [Rescan
> (parallel) , 0.0075330 secs]1086.029: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 79381K(139444K), 0.0076440 secs]
> [Times: user=0.09 sys=0.01, real=0.01 secs]
> 1086.029: [CMS-concurrent-sweep-start]
> 1086.031: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1086.031: [CMS-concurrent-reset-start]
> 1086.041: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1088.041: [GC [1 CMS-initial-mark: 12849K(21428K)] 79509K(139444K),
> 0.0091350 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1088.050: [CMS-concurrent-mark-start]
> 1088.066: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1088.067: [CMS-concurrent-preclean-start]
> 1088.067: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1088.067: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1093.070:
> [CMS-concurrent-abortable-preclean: 0.698/5.004 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1093.071: [GC[YG occupancy: 66980 K (118016 K)]1093.071: [Rescan
> (parallel) , 0.0051870 secs]1093.076: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 79830K(139444K), 0.0052930 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 1093.076: [CMS-concurrent-sweep-start]
> 1093.078: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1093.078: [CMS-concurrent-reset-start]
> 1093.087: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1095.088: [GC [1 CMS-initial-mark: 12849K(21428K)] 79958K(139444K),
> 0.0091350 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1095.097: [CMS-concurrent-mark-start]
> 1095.114: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1095.114: [CMS-concurrent-preclean-start]
> 1095.115: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1095.115: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1100.121:
> [CMS-concurrent-abortable-preclean: 0.701/5.007 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 1100.121: [GC[YG occupancy: 67428 K (118016 K)]1100.122: [Rescan
> (parallel) , 0.0068510 secs]1100.128: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 80278K(139444K), 0.0069510 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1100.129: [CMS-concurrent-sweep-start]
> 1100.130: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1100.130: [CMS-concurrent-reset-start]
> 1100.138: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1102.139: [GC [1 CMS-initial-mark: 12849K(21428K)] 80406K(139444K),
> 0.0090760 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1102.148: [CMS-concurrent-mark-start]
> 1102.165: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1102.165: [CMS-concurrent-preclean-start]
> 1102.165: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1102.165: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1107.168:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1107.168: [GC[YG occupancy: 67876 K (118016 K)]1107.168: [Rescan
> (parallel) , 0.0076420 secs]1107.176: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 80726K(139444K), 0.0077500 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1107.176: [CMS-concurrent-sweep-start]
> 1107.178: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1107.178: [CMS-concurrent-reset-start]
> 1107.187: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1109.188: [GC [1 CMS-initial-mark: 12849K(21428K)] 80854K(139444K),
> 0.0091510 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1109.197: [CMS-concurrent-mark-start]
> 1109.214: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1109.214: [CMS-concurrent-preclean-start]
> 1109.214: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1109.214: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1114.290:
> [CMS-concurrent-abortable-preclean: 0.711/5.076 secs] [Times:
> user=0.72 sys=0.00, real=5.07 secs]
> 1114.290: [GC[YG occupancy: 68473 K (118016 K)]1114.290: [Rescan
> (parallel) , 0.0084730 secs]1114.299: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 81322K(139444K), 0.0085810 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1114.299: [CMS-concurrent-sweep-start]
> 1114.301: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1114.301: [CMS-concurrent-reset-start]
> 1114.310: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1115.803: [GC [1 CMS-initial-mark: 12849K(21428K)] 81451K(139444K),
> 0.0106050 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1115.814: [CMS-concurrent-mark-start]
> 1115.830: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1115.830: [CMS-concurrent-preclean-start]
> 1115.831: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1115.831: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1120.839:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1120.839: [GC[YG occupancy: 68921 K (118016 K)]1120.839: [Rescan
> (parallel) , 0.0088800 secs]1120.848: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 81771K(139444K), 0.0089910 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1120.848: [CMS-concurrent-sweep-start]
> 1120.850: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1120.850: [CMS-concurrent-reset-start]
> 1120.858: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1122.859: [GC [1 CMS-initial-mark: 12849K(21428K)] 81899K(139444K),
> 0.0092280 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1122.868: [CMS-concurrent-mark-start]
> 1122.885: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1122.885: [CMS-concurrent-preclean-start]
> 1122.885: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1122.885: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1127.888:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 1127.888: [GC[YG occupancy: 69369 K (118016 K)]1127.888: [Rescan
> (parallel) , 0.0087740 secs]1127.897: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 82219K(139444K), 0.0088850 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1127.897: [CMS-concurrent-sweep-start]
> 1127.898: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1127.898: [CMS-concurrent-reset-start]
> 1127.906: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1129.907: [GC [1 CMS-initial-mark: 12849K(21428K)] 82347K(139444K),
> 0.0092280 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1129.916: [CMS-concurrent-mark-start]
> 1129.933: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1129.933: [CMS-concurrent-preclean-start]
> 1129.934: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1129.934: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1134.938:
> [CMS-concurrent-abortable-preclean: 0.699/5.004 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1134.938: [GC[YG occupancy: 69818 K (118016 K)]1134.939: [Rescan
> (parallel) , 0.0078530 secs]1134.946: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 82667K(139444K), 0.0079630 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1134.947: [CMS-concurrent-sweep-start]
> 1134.948: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1134.948: [CMS-concurrent-reset-start]
> 1134.956: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1136.957: [GC [1 CMS-initial-mark: 12849K(21428K)] 82795K(139444K),
> 0.0092760 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1136.966: [CMS-concurrent-mark-start]
> 1136.983: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1136.983: [CMS-concurrent-preclean-start]
> 1136.984: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.01 secs]
> 1136.984: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1141.991:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1141.991: [GC[YG occupancy: 70266 K (118016 K)]1141.991: [Rescan
> (parallel) , 0.0090620 secs]1142.000: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 83116K(139444K), 0.0091700 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1142.000: [CMS-concurrent-sweep-start]
> 1142.002: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1142.002: [CMS-concurrent-reset-start]
> 1142.011: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1142.657: [GC [1 CMS-initial-mark: 12849K(21428K)] 83390K(139444K),
> 0.0094330 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1142.667: [CMS-concurrent-mark-start]
> 1142.685: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1142.685: [CMS-concurrent-preclean-start]
> 1142.686: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1142.686: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1147.688:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1147.688: [GC[YG occupancy: 70901 K (118016 K)]1147.688: [Rescan
> (parallel) , 0.0081170 secs]1147.696: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 83751K(139444K), 0.0082390 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1147.697: [CMS-concurrent-sweep-start]
> 1147.698: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1147.698: [CMS-concurrent-reset-start]
> 1147.706: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1149.706: [GC [1 CMS-initial-mark: 12849K(21428K)] 83879K(139444K),
> 0.0095560 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1149.716: [CMS-concurrent-mark-start]
> 1149.734: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1149.734: [CMS-concurrent-preclean-start]
> 1149.734: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1149.734: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1154.741:
> [CMS-concurrent-abortable-preclean: 0.701/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1154.741: [GC[YG occupancy: 71349 K (118016 K)]1154.741: [Rescan
> (parallel) , 0.0090720 secs]1154.750: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 84199K(139444K), 0.0091780 secs]
> [Times: user=0.10 sys=0.01, real=0.01 secs]
> 1154.750: [CMS-concurrent-sweep-start]
> 1154.752: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1154.752: [CMS-concurrent-reset-start]
> 1154.762: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1155.021: [GC [1 CMS-initial-mark: 12849K(21428K)] 84199K(139444K),
> 0.0094030 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1155.031: [CMS-concurrent-mark-start]
> 1155.047: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1155.047: [CMS-concurrent-preclean-start]
> 1155.047: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1155.047: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1160.056:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1160.056: [GC[YG occupancy: 71669 K (118016 K)]1160.056: [Rescan
> (parallel) , 0.0056520 secs]1160.062: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 84519K(139444K), 0.0057790 secs]
> [Times: user=0.07 sys=0.00, real=0.00 secs]
> 1160.062: [CMS-concurrent-sweep-start]
> 1160.064: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1160.064: [CMS-concurrent-reset-start]
> 1160.073: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1162.074: [GC [1 CMS-initial-mark: 12849K(21428K)] 84647K(139444K),
> 0.0095040 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1162.083: [CMS-concurrent-mark-start]
> 1162.098: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1162.098: [CMS-concurrent-preclean-start]
> 1162.099: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1162.099: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1167.102:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1167.102: [GC[YG occupancy: 72118 K (118016 K)]1167.102: [Rescan
> (parallel) , 0.0072180 secs]1167.110: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 84968K(139444K), 0.0073300 secs]
> [Times: user=0.08 sys=0.00, real=0.01 secs]
> 1167.110: [CMS-concurrent-sweep-start]
> 1167.112: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1167.112: [CMS-concurrent-reset-start]
> 1167.121: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1169.121: [GC [1 CMS-initial-mark: 12849K(21428K)] 85096K(139444K),
> 0.0096940 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1169.131: [CMS-concurrent-mark-start]
> 1169.147: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1169.147: [CMS-concurrent-preclean-start]
> 1169.147: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1169.147: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1174.197:
> [CMS-concurrent-abortable-preclean: 0.720/5.050 secs] [Times:
> user=0.72 sys=0.01, real=5.05 secs]
> 1174.198: [GC[YG occupancy: 72607 K (118016 K)]1174.198: [Rescan
> (parallel) , 0.0064910 secs]1174.204: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 85456K(139444K), 0.0065940 secs]
> [Times: user=0.06 sys=0.01, real=0.01 secs]
> 1174.204: [CMS-concurrent-sweep-start]
> 1174.206: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1174.206: [CMS-concurrent-reset-start]
> 1174.215: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1176.215: [GC [1 CMS-initial-mark: 12849K(21428K)] 85585K(139444K),
> 0.0095940 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1176.225: [CMS-concurrent-mark-start]
> 1176.240: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1176.240: [CMS-concurrent-preclean-start]
> 1176.241: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1176.241: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1181.244:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1181.244: [GC[YG occupancy: 73055 K (118016 K)]1181.244: [Rescan
> (parallel) , 0.0093030 secs]1181.254: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 85905K(139444K), 0.0094040 secs]
> [Times: user=0.09 sys=0.01, real=0.01 secs]
> 1181.254: [CMS-concurrent-sweep-start]
> 1181.256: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1181.256: [CMS-concurrent-reset-start]
> 1181.265: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1183.266: [GC [1 CMS-initial-mark: 12849K(21428K)] 86033K(139444K),
> 0.0096490 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1183.275: [CMS-concurrent-mark-start]
> 1183.293: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.08
> sys=0.00, real=0.02 secs]
> 1183.293: [CMS-concurrent-preclean-start]
> 1183.294: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1183.294: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1188.301:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1188.301: [GC[YG occupancy: 73503 K (118016 K)]1188.301: [Rescan
> (parallel) , 0.0092610 secs]1188.310: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 86353K(139444K), 0.0093750 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1188.310: [CMS-concurrent-sweep-start]
> 1188.312: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1188.312: [CMS-concurrent-reset-start]
> 1188.320: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1190.321: [GC [1 CMS-initial-mark: 12849K(21428K)] 86481K(139444K),
> 0.0097510 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1190.331: [CMS-concurrent-mark-start]
> 1190.347: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1190.347: [CMS-concurrent-preclean-start]
> 1190.347: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1190.347: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1195.359:
> [CMS-concurrent-abortable-preclean: 0.706/5.011 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1195.359: [GC[YG occupancy: 73952 K (118016 K)]1195.359: [Rescan
> (parallel) , 0.0093210 secs]1195.368: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 86801K(139444K), 0.0094330 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1195.369: [CMS-concurrent-sweep-start]
> 1195.370: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1195.370: [CMS-concurrent-reset-start]
> 1195.378: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1196.543: [GC [1 CMS-initial-mark: 12849K(21428K)] 88001K(139444K),
> 0.0099870 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1196.553: [CMS-concurrent-mark-start]
> 1196.570: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1196.570: [CMS-concurrent-preclean-start]
> 1196.570: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1196.570: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1201.574:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1201.574: [GC[YG occupancy: 75472 K (118016 K)]1201.574: [Rescan
> (parallel) , 0.0096480 secs]1201.584: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 88322K(139444K), 0.0097500 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1201.584: [CMS-concurrent-sweep-start]
> 1201.586: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1201.586: [CMS-concurrent-reset-start]
> 1201.595: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1202.679: [GC [1 CMS-initial-mark: 12849K(21428K)] 88491K(139444K),
> 0.0099400 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1202.690: [CMS-concurrent-mark-start]
> 1202.708: [CMS-concurrent-mark: 0.016/0.019 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1202.708: [CMS-concurrent-preclean-start]
> 1202.709: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1202.709: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1207.718:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1207.718: [GC[YG occupancy: 76109 K (118016 K)]1207.718: [Rescan
> (parallel) , 0.0096360 secs]1207.727: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 88959K(139444K), 0.0097380 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1207.728: [CMS-concurrent-sweep-start]
> 1207.729: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1207.729: [CMS-concurrent-reset-start]
> 1207.737: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1209.738: [GC [1 CMS-initial-mark: 12849K(21428K)] 89087K(139444K),
> 0.0099440 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1209.748: [CMS-concurrent-mark-start]
> 1209.765: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1209.765: [CMS-concurrent-preclean-start]
> 1209.765: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1209.765: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1214.797:
> [CMS-concurrent-abortable-preclean: 0.716/5.031 secs] [Times:
> user=0.72 sys=0.00, real=5.03 secs]
> 1214.797: [GC[YG occupancy: 76557 K (118016 K)]1214.797: [Rescan
> (parallel) , 0.0096280 secs]1214.807: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 89407K(139444K), 0.0097320 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1214.807: [CMS-concurrent-sweep-start]
> 1214.808: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1214.808: [CMS-concurrent-reset-start]
> 1214.816: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1216.817: [GC [1 CMS-initial-mark: 12849K(21428K)] 89535K(139444K),
> 0.0099640 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1216.827: [CMS-concurrent-mark-start]
> 1216.844: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1216.844: [CMS-concurrent-preclean-start]
> 1216.844: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1216.844: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1221.847:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1221.847: [GC[YG occupancy: 77005 K (118016 K)]1221.847: [Rescan
> (parallel) , 0.0061810 secs]1221.854: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 89855K(139444K), 0.0062950 secs]
> [Times: user=0.07 sys=0.00, real=0.01 secs]
> 1221.854: [CMS-concurrent-sweep-start]
> 1221.855: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1221.855: [CMS-concurrent-reset-start]
> 1221.864: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1223.865: [GC [1 CMS-initial-mark: 12849K(21428K)] 89983K(139444K),
> 0.0100430 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1223.875: [CMS-concurrent-mark-start]
> 1223.890: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1223.890: [CMS-concurrent-preclean-start]
> 1223.891: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1223.891: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1228.899:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1228.899: [GC[YG occupancy: 77454 K (118016 K)]1228.899: [Rescan
> (parallel) , 0.0095850 secs]1228.909: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 90304K(139444K), 0.0096960 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1228.909: [CMS-concurrent-sweep-start]
> 1228.911: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1228.911: [CMS-concurrent-reset-start]
> 1228.919: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1230.919: [GC [1 CMS-initial-mark: 12849K(21428K)] 90432K(139444K),
> 0.0101360 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1230.930: [CMS-concurrent-mark-start]
> 1230.946: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1230.946: [CMS-concurrent-preclean-start]
> 1230.947: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1230.947: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1235.952:
> [CMS-concurrent-abortable-preclean: 0.699/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1235.952: [GC[YG occupancy: 77943 K (118016 K)]1235.952: [Rescan
> (parallel) , 0.0084420 secs]1235.961: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 90793K(139444K), 0.0085450 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1235.961: [CMS-concurrent-sweep-start]
> 1235.963: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1235.963: [CMS-concurrent-reset-start]
> 1235.972: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1237.973: [GC [1 CMS-initial-mark: 12849K(21428K)] 90921K(139444K),
> 0.0101280 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1237.983: [CMS-concurrent-mark-start]
> 1237.998: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1237.998: [CMS-concurrent-preclean-start]
> 1237.999: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1237.999: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1243.008:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1243.008: [GC[YG occupancy: 78391 K (118016 K)]1243.008: [Rescan
> (parallel) , 0.0090510 secs]1243.017: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 91241K(139444K), 0.0091560 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1243.017: [CMS-concurrent-sweep-start]
> 1243.019: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1243.019: [CMS-concurrent-reset-start]
> 1243.027: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1245.027: [GC [1 CMS-initial-mark: 12849K(21428K)] 91369K(139444K),
> 0.0101080 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1245.038: [CMS-concurrent-mark-start]
> 1245.055: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1245.055: [CMS-concurrent-preclean-start]
> 1245.055: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1245.055: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1250.058:
> [CMS-concurrent-abortable-preclean: 0.697/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1250.058: [GC[YG occupancy: 78839 K (118016 K)]1250.058: [Rescan
> (parallel) , 0.0096920 secs]1250.068: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 91689K(139444K), 0.0098040 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1250.068: [CMS-concurrent-sweep-start]
> 1250.070: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1250.070: [CMS-concurrent-reset-start]
> 1250.078: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1252.078: [GC [1 CMS-initial-mark: 12849K(21428K)] 91817K(139444K),
> 0.0102560 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1252.089: [CMS-concurrent-mark-start]
> 1252.105: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1252.105: [CMS-concurrent-preclean-start]
> 1252.106: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1252.106: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1257.113:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1257.113: [GC[YG occupancy: 79288 K (118016 K)]1257.113: [Rescan
> (parallel) , 0.0089920 secs]1257.122: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 92137K(139444K), 0.0090960 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1257.122: [CMS-concurrent-sweep-start]
> 1257.124: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1257.124: [CMS-concurrent-reset-start]
> 1257.133: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1259.134: [GC [1 CMS-initial-mark: 12849K(21428K)] 92266K(139444K),
> 0.0101720 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1259.144: [CMS-concurrent-mark-start]
> 1259.159: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 1259.159: [CMS-concurrent-preclean-start]
> 1259.159: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1259.159: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1264.229:
> [CMS-concurrent-abortable-preclean: 0.716/5.070 secs] [Times:
> user=0.72 sys=0.01, real=5.07 secs]
> 1264.229: [GC[YG occupancy: 79881 K (118016 K)]1264.229: [Rescan
> (parallel) , 0.0101320 secs]1264.240: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 92731K(139444K), 0.0102440 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1264.240: [CMS-concurrent-sweep-start]
> 1264.241: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1264.241: [CMS-concurrent-reset-start]
> 1264.250: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1266.250: [GC [1 CMS-initial-mark: 12849K(21428K)] 92859K(139444K),
> 0.0105180 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1266.261: [CMS-concurrent-mark-start]
> 1266.277: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1266.277: [CMS-concurrent-preclean-start]
> 1266.277: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1266.277: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1271.285:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1271.285: [GC[YG occupancy: 80330 K (118016 K)]1271.285: [Rescan
> (parallel) , 0.0094600 secs]1271.295: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 93180K(139444K), 0.0095600 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1271.295: [CMS-concurrent-sweep-start]
> 1271.297: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1271.297: [CMS-concurrent-reset-start]
> 1271.306: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1273.306: [GC [1 CMS-initial-mark: 12849K(21428K)] 93308K(139444K),
> 0.0104100 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1273.317: [CMS-concurrent-mark-start]
> 1273.334: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1273.334: [CMS-concurrent-preclean-start]
> 1273.335: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1273.335: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1278.341:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1278.341: [GC[YG occupancy: 80778 K (118016 K)]1278.341: [Rescan
> (parallel) , 0.0101320 secs]1278.351: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 93628K(139444K), 0.0102460 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1278.351: [CMS-concurrent-sweep-start]
> 1278.353: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1278.353: [CMS-concurrent-reset-start]
> 1278.362: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1280.362: [GC [1 CMS-initial-mark: 12849K(21428K)] 93756K(139444K),
> 0.0105680 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1280.373: [CMS-concurrent-mark-start]
> 1280.388: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1280.388: [CMS-concurrent-preclean-start]
> 1280.388: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1280.388: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1285.400:
> [CMS-concurrent-abortable-preclean: 0.706/5.012 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1285.400: [GC[YG occupancy: 81262 K (118016 K)]1285.400: [Rescan
> (parallel) , 0.0093660 secs]1285.410: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 94111K(139444K), 0.0094820 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1285.410: [CMS-concurrent-sweep-start]
> 1285.411: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1285.411: [CMS-concurrent-reset-start]
> 1285.420: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1287.420: [GC [1 CMS-initial-mark: 12849K(21428K)] 94240K(139444K),
> 0.0105800 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1287.431: [CMS-concurrent-mark-start]
> 1287.447: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1287.447: [CMS-concurrent-preclean-start]
> 1287.447: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1287.447: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1292.460:
> [CMS-concurrent-abortable-preclean: 0.707/5.012 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1292.460: [GC[YG occupancy: 81710 K (118016 K)]1292.460: [Rescan
> (parallel) , 0.0081130 secs]1292.468: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 94560K(139444K), 0.0082210 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1292.468: [CMS-concurrent-sweep-start]
> 1292.470: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1292.470: [CMS-concurrent-reset-start]
> 1292.480: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1292.712: [GC [1 CMS-initial-mark: 12849K(21428K)] 94624K(139444K),
> 0.0104870 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1292.723: [CMS-concurrent-mark-start]
> 1292.739: [CMS-concurrent-mark: 0.015/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1292.739: [CMS-concurrent-preclean-start]
> 1292.740: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1292.740: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1297.748:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1297.748: [GC[YG occupancy: 82135 K (118016 K)]1297.748: [Rescan
> (parallel) , 0.0106180 secs]1297.759: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 94985K(139444K), 0.0107410 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1297.759: [CMS-concurrent-sweep-start]
> 1297.760: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1297.761: [CMS-concurrent-reset-start]
> 1297.769: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1299.769: [GC [1 CMS-initial-mark: 12849K(21428K)] 95113K(139444K),
> 0.0105340 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1299.780: [CMS-concurrent-mark-start]
> 1299.796: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1299.796: [CMS-concurrent-preclean-start]
> 1299.797: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1299.797: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1304.805:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.69 sys=0.00, real=5.01 secs]
> 1304.805: [GC[YG occupancy: 82583 K (118016 K)]1304.806: [Rescan
> (parallel) , 0.0094010 secs]1304.815: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 95433K(139444K), 0.0095140 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1304.815: [CMS-concurrent-sweep-start]
> 1304.817: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1304.817: [CMS-concurrent-reset-start]
> 1304.827: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1306.827: [GC [1 CMS-initial-mark: 12849K(21428K)] 95561K(139444K),
> 0.0107300 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1306.838: [CMS-concurrent-mark-start]
> 1306.855: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1306.855: [CMS-concurrent-preclean-start]
> 1306.855: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1306.855: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1311.858:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1311.858: [GC[YG occupancy: 83032 K (118016 K)]1311.858: [Rescan
> (parallel) , 0.0094210 secs]1311.867: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 95882K(139444K), 0.0095360 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1311.868: [CMS-concurrent-sweep-start]
> 1311.869: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1311.869: [CMS-concurrent-reset-start]
> 1311.877: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1313.878: [GC [1 CMS-initial-mark: 12849K(21428K)] 96010K(139444K),
> 0.0107870 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1313.889: [CMS-concurrent-mark-start]
> 1313.905: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1313.905: [CMS-concurrent-preclean-start]
> 1313.906: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1313.906: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1318.914:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1318.915: [GC[YG occupancy: 83481 K (118016 K)]1318.915: [Rescan
> (parallel) , 0.0096280 secs]1318.924: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 96331K(139444K), 0.0097340 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1318.925: [CMS-concurrent-sweep-start]
> 1318.927: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1318.927: [CMS-concurrent-reset-start]
> 1318.936: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1320.936: [GC [1 CMS-initial-mark: 12849K(21428K)] 96459K(139444K),
> 0.0106300 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1320.947: [CMS-concurrent-mark-start]
> 1320.964: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1320.964: [CMS-concurrent-preclean-start]
> 1320.965: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1320.965: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1325.991:
> [CMS-concurrent-abortable-preclean: 0.717/5.026 secs] [Times:
> user=0.73 sys=0.00, real=5.02 secs]
> 1325.991: [GC[YG occupancy: 84205 K (118016 K)]1325.991: [Rescan
> (parallel) , 0.0097880 secs]1326.001: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 97055K(139444K), 0.0099010 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1326.001: [CMS-concurrent-sweep-start]
> 1326.003: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1326.003: [CMS-concurrent-reset-start]
> 1326.012: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1328.013: [GC [1 CMS-initial-mark: 12849K(21428K)] 97183K(139444K),
> 0.0109730 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1328.024: [CMS-concurrent-mark-start]
> 1328.039: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1328.039: [CMS-concurrent-preclean-start]
> 1328.039: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1328.039: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1333.043:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1333.043: [GC[YG occupancy: 84654 K (118016 K)]1333.043: [Rescan
> (parallel) , 0.0110740 secs]1333.054: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 97504K(139444K), 0.0111760 secs]
> [Times: user=0.12 sys=0.01, real=0.02 secs]
> 1333.054: [CMS-concurrent-sweep-start]
> 1333.056: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1333.056: [CMS-concurrent-reset-start]
> 1333.065: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1335.066: [GC [1 CMS-initial-mark: 12849K(21428K)] 97632K(139444K),
> 0.0109300 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1335.077: [CMS-concurrent-mark-start]
> 1335.094: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1335.094: [CMS-concurrent-preclean-start]
> 1335.094: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1335.094: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1340.103:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1340.103: [GC[YG occupancy: 85203 K (118016 K)]1340.103: [Rescan
> (parallel) , 0.0109470 secs]1340.114: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 98052K(139444K), 0.0110500 secs]
> [Times: user=0.11 sys=0.01, real=0.02 secs]
> 1340.114: [CMS-concurrent-sweep-start]
> 1340.116: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1340.116: [CMS-concurrent-reset-start]
> 1340.125: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1342.126: [GC [1 CMS-initial-mark: 12849K(21428K)] 98181K(139444K),
> 0.0109170 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1342.137: [CMS-concurrent-mark-start]
> 1342.154: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1342.154: [CMS-concurrent-preclean-start]
> 1342.154: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1342.154: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1347.161:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1347.162: [GC[YG occupancy: 85652 K (118016 K)]1347.162: [Rescan
> (parallel) , 0.0075610 secs]1347.169: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 98502K(139444K), 0.0076680 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1347.169: [CMS-concurrent-sweep-start]
> 1347.171: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1347.172: [CMS-concurrent-reset-start]
> 1347.181: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1349.181: [GC [1 CMS-initial-mark: 12849K(21428K)] 98630K(139444K),
> 0.0109540 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1349.192: [CMS-concurrent-mark-start]
> 1349.208: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1349.208: [CMS-concurrent-preclean-start]
> 1349.208: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1349.208: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1354.268:
> [CMS-concurrent-abortable-preclean: 0.723/5.060 secs] [Times:
> user=0.73 sys=0.00, real=5.06 secs]
> 1354.268: [GC[YG occupancy: 86241 K (118016 K)]1354.268: [Rescan
> (parallel) , 0.0099530 secs]1354.278: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 99091K(139444K), 0.0100670 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1354.278: [CMS-concurrent-sweep-start]
> 1354.280: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1354.280: [CMS-concurrent-reset-start]
> 1354.288: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1356.289: [GC [1 CMS-initial-mark: 12849K(21428K)] 99219K(139444K),
> 0.0111450 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1356.300: [CMS-concurrent-mark-start]
> 1356.316: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1356.316: [CMS-concurrent-preclean-start]
> 1356.317: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1356.317: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1361.322:
> [CMS-concurrent-abortable-preclean: 0.700/5.005 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1361.322: [GC[YG occupancy: 86690 K (118016 K)]1361.322: [Rescan
> (parallel) , 0.0097180 secs]1361.332: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 99540K(139444K), 0.0098210 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1361.332: [CMS-concurrent-sweep-start]
> 1361.333: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1361.333: [CMS-concurrent-reset-start]
> 1361.342: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1363.342: [GC [1 CMS-initial-mark: 12849K(21428K)] 99668K(139444K),
> 0.0110230 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1363.354: [CMS-concurrent-mark-start]
> 1363.368: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1363.368: [CMS-concurrent-preclean-start]
> 1363.369: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1363.369: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1368.378:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1368.378: [GC[YG occupancy: 87139 K (118016 K)]1368.378: [Rescan
> (parallel) , 0.0100770 secs]1368.388: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 99989K(139444K), 0.0101900 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1368.388: [CMS-concurrent-sweep-start]
> 1368.390: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1368.390: [CMS-concurrent-reset-start]
> 1368.398: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1370.399: [GC [1 CMS-initial-mark: 12849K(21428K)] 100117K(139444K),
> 0.0111810 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1370.410: [CMS-concurrent-mark-start]
> 1370.426: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1370.426: [CMS-concurrent-preclean-start]
> 1370.427: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1370.427: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1375.447:
> [CMS-concurrent-abortable-preclean: 0.715/5.020 secs] [Times:
> user=0.72 sys=0.00, real=5.02 secs]
> 1375.447: [GC[YG occupancy: 87588 K (118016 K)]1375.447: [Rescan
> (parallel) , 0.0101690 secs]1375.457: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 100438K(139444K), 0.0102730 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1375.457: [CMS-concurrent-sweep-start]
> 1375.459: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1375.459: [CMS-concurrent-reset-start]
> 1375.467: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1377.467: [GC [1 CMS-initial-mark: 12849K(21428K)] 100566K(139444K),
> 0.0110760 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1377.478: [CMS-concurrent-mark-start]
> 1377.495: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1377.495: [CMS-concurrent-preclean-start]
> 1377.496: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1377.496: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1382.502:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 1382.502: [GC[YG occupancy: 89213 K (118016 K)]1382.502: [Rescan
> (parallel) , 0.0108630 secs]1382.513: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 102063K(139444K), 0.0109700 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1382.513: [CMS-concurrent-sweep-start]
> 1382.514: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1382.514: [CMS-concurrent-reset-start]
> 1382.523: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1382.743: [GC [1 CMS-initial-mark: 12849K(21428K)] 102127K(139444K),
> 0.0113140 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1382.755: [CMS-concurrent-mark-start]
> 1382.773: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1382.773: [CMS-concurrent-preclean-start]
> 1382.774: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1382.774: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1387.777:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1387.777: [GC[YG occupancy: 89638 K (118016 K)]1387.777: [Rescan
> (parallel) , 0.0113310 secs]1387.789: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 102488K(139444K), 0.0114430 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1387.789: [CMS-concurrent-sweep-start]
> 1387.790: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1387.790: [CMS-concurrent-reset-start]
> 1387.799: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1389.799: [GC [1 CMS-initial-mark: 12849K(21428K)] 102617K(139444K),
> 0.0113540 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1389.810: [CMS-concurrent-mark-start]
> 1389.827: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1389.827: [CMS-concurrent-preclean-start]
> 1389.827: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1389.827: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1394.831:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1394.831: [GC[YG occupancy: 90088 K (118016 K)]1394.831: [Rescan
> (parallel) , 0.0103790 secs]1394.841: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 102938K(139444K), 0.0104960 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1394.842: [CMS-concurrent-sweep-start]
> 1394.844: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1394.844: [CMS-concurrent-reset-start]
> 1394.853: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1396.853: [GC [1 CMS-initial-mark: 12849K(21428K)] 103066K(139444K),
> 0.0114740 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1396.865: [CMS-concurrent-mark-start]
> 1396.880: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1396.880: [CMS-concurrent-preclean-start]
> 1396.881: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1396.881: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1401.890:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1401.890: [GC[YG occupancy: 90537 K (118016 K)]1401.891: [Rescan
> (parallel) , 0.0116110 secs]1401.902: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 103387K(139444K), 0.0117240 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1401.902: [CMS-concurrent-sweep-start]
> 1401.904: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1401.904: [CMS-concurrent-reset-start]
> 1401.914: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1403.914: [GC [1 CMS-initial-mark: 12849K(21428K)] 103515K(139444K),
> 0.0111980 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1403.925: [CMS-concurrent-mark-start]
> 1403.943: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1403.943: [CMS-concurrent-preclean-start]
> 1403.944: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.01 secs]
> 1403.944: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1408.982:
> [CMS-concurrent-abortable-preclean: 0.718/5.038 secs] [Times:
> user=0.72 sys=0.00, real=5.03 secs]
> 1408.982: [GC[YG occupancy: 90986 K (118016 K)]1408.982: [Rescan
> (parallel) , 0.0115260 secs]1408.994: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 103836K(139444K), 0.0116320 secs]
> [Times: user=0.13 sys=0.00, real=0.02 secs]
> 1408.994: [CMS-concurrent-sweep-start]
> 1408.996: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1408.996: [CMS-concurrent-reset-start]
> 1409.005: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1411.005: [GC [1 CMS-initial-mark: 12849K(21428K)] 103964K(139444K),
> 0.0114590 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1411.017: [CMS-concurrent-mark-start]
> 1411.034: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1411.034: [CMS-concurrent-preclean-start]
> 1411.034: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1411.034: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1416.140:
> [CMS-concurrent-abortable-preclean: 0.712/5.105 secs] [Times:
> user=0.71 sys=0.00, real=5.10 secs]
> 1416.140: [GC[YG occupancy: 91476 K (118016 K)]1416.140: [Rescan
> (parallel) , 0.0114950 secs]1416.152: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 104326K(139444K), 0.0116020 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1416.152: [CMS-concurrent-sweep-start]
> 1416.154: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1416.154: [CMS-concurrent-reset-start]
> 1416.163: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1418.163: [GC [1 CMS-initial-mark: 12849K(21428K)] 104454K(139444K),
> 0.0114040 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1418.175: [CMS-concurrent-mark-start]
> 1418.191: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1418.191: [CMS-concurrent-preclean-start]
> 1418.191: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1418.191: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1423.198:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1423.199: [GC[YG occupancy: 91925 K (118016 K)]1423.199: [Rescan
> (parallel) , 0.0105460 secs]1423.209: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 104775K(139444K), 0.0106640 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1423.209: [CMS-concurrent-sweep-start]
> 1423.211: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1423.211: [CMS-concurrent-reset-start]
> 1423.220: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1425.220: [GC [1 CMS-initial-mark: 12849K(21428K)] 104903K(139444K),
> 0.0116300 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1425.232: [CMS-concurrent-mark-start]
> 1425.248: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1425.248: [CMS-concurrent-preclean-start]
> 1425.248: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1425.248: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1430.252:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1430.252: [GC[YG occupancy: 92374 K (118016 K)]1430.252: [Rescan
> (parallel) , 0.0098720 secs]1430.262: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 105224K(139444K), 0.0099750 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1430.262: [CMS-concurrent-sweep-start]
> 1430.264: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1430.264: [CMS-concurrent-reset-start]
> 1430.273: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1432.274: [GC [1 CMS-initial-mark: 12849K(21428K)] 105352K(139444K),
> 0.0114050 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1432.285: [CMS-concurrent-mark-start]
> 1432.301: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1432.301: [CMS-concurrent-preclean-start]
> 1432.301: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1432.301: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1437.304:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1437.305: [GC[YG occupancy: 92823 K (118016 K)]1437.305: [Rescan
> (parallel) , 0.0115010 secs]1437.316: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 105673K(139444K), 0.0116090 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1437.316: [CMS-concurrent-sweep-start]
> 1437.319: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1437.319: [CMS-concurrent-reset-start]
> 1437.328: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1439.328: [GC [1 CMS-initial-mark: 12849K(21428K)] 105801K(139444K),
> 0.0115740 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1439.340: [CMS-concurrent-mark-start]
> 1439.356: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1439.356: [CMS-concurrent-preclean-start]
> 1439.356: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1439.356: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1444.411:
> [CMS-concurrent-abortable-preclean: 0.715/5.054 secs] [Times:
> user=0.72 sys=0.00, real=5.05 secs]
> 1444.411: [GC[YG occupancy: 93547 K (118016 K)]1444.411: [Rescan
> (parallel) , 0.0072910 secs]1444.418: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 106397K(139444K), 0.0073970 secs]
> [Times: user=0.09 sys=0.00, real=0.01 secs]
> 1444.419: [CMS-concurrent-sweep-start]
> 1444.420: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1444.420: [CMS-concurrent-reset-start]
> 1444.429: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1446.429: [GC [1 CMS-initial-mark: 12849K(21428K)] 106525K(139444K),
> 0.0117950 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1446.441: [CMS-concurrent-mark-start]
> 1446.457: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1446.457: [CMS-concurrent-preclean-start]
> 1446.458: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1446.458: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1451.461:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1451.461: [GC[YG occupancy: 93996 K (118016 K)]1451.461: [Rescan
> (parallel) , 0.0120870 secs]1451.473: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 106846K(139444K), 0.0121920 secs]
> [Times: user=0.14 sys=0.00, real=0.02 secs]
> 1451.473: [CMS-concurrent-sweep-start]
> 1451.476: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1451.476: [CMS-concurrent-reset-start]
> 1451.485: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1453.485: [GC [1 CMS-initial-mark: 12849K(21428K)] 106974K(139444K),
> 0.0117990 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1453.497: [CMS-concurrent-mark-start]
> 1453.514: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1453.514: [CMS-concurrent-preclean-start]
> 1453.515: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1453.515: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1458.518:
> [CMS-concurrent-abortable-preclean: 0.697/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1458.518: [GC[YG occupancy: 94445 K (118016 K)]1458.518: [Rescan
> (parallel) , 0.0123720 secs]1458.530: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 107295K(139444K), 0.0124750 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1458.530: [CMS-concurrent-sweep-start]
> 1458.532: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1458.532: [CMS-concurrent-reset-start]
> 1458.540: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1460.541: [GC [1 CMS-initial-mark: 12849K(21428K)] 107423K(139444K),
> 0.0118680 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1460.553: [CMS-concurrent-mark-start]
> 1460.568: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1460.568: [CMS-concurrent-preclean-start]
> 1460.569: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1460.569: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1465.577:
> [CMS-concurrent-abortable-preclean: 0.703/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1465.577: [GC[YG occupancy: 94894 K (118016 K)]1465.577: [Rescan
> (parallel) , 0.0119100 secs]1465.589: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 107744K(139444K), 0.0120270 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1465.590: [CMS-concurrent-sweep-start]
> 1465.591: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1465.591: [CMS-concurrent-reset-start]
> 1465.600: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1467.600: [GC [1 CMS-initial-mark: 12849K(21428K)] 107937K(139444K),
> 0.0120020 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1467.612: [CMS-concurrent-mark-start]
> 1467.628: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1467.628: [CMS-concurrent-preclean-start]
> 1467.628: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1467.628: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1472.636:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1472.637: [GC[YG occupancy: 95408 K (118016 K)]1472.637: [Rescan
> (parallel) , 0.0119090 secs]1472.649: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 108257K(139444K), 0.0120260 secs]
> [Times: user=0.13 sys=0.00, real=0.01 secs]
> 1472.649: [CMS-concurrent-sweep-start]
> 1472.650: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1472.650: [CMS-concurrent-reset-start]
> 1472.659: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1472.775: [GC [1 CMS-initial-mark: 12849K(21428K)] 108365K(139444K),
> 0.0120260 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1472.787: [CMS-concurrent-mark-start]
> 1472.805: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1472.805: [CMS-concurrent-preclean-start]
> 1472.806: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.01 sys=0.00, real=0.00 secs]
> 1472.806: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1477.808:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1477.808: [GC[YG occupancy: 95876 K (118016 K)]1477.808: [Rescan
> (parallel) , 0.0099490 secs]1477.818: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 108726K(139444K), 0.0100580 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1477.818: [CMS-concurrent-sweep-start]
> 1477.820: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1477.820: [CMS-concurrent-reset-start]
> 1477.828: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1479.829: [GC [1 CMS-initial-mark: 12849K(21428K)] 108854K(139444K),
> 0.0119550 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1479.841: [CMS-concurrent-mark-start]
> 1479.857: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1479.857: [CMS-concurrent-preclean-start]
> 1479.857: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1479.857: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1484.870:
> [CMS-concurrent-abortable-preclean: 0.707/5.012 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1484.870: [GC[YG occupancy: 96325 K (118016 K)]1484.870: [Rescan
> (parallel) , 0.0122870 secs]1484.882: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 109175K(139444K), 0.0123900 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1484.882: [CMS-concurrent-sweep-start]
> 1484.884: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1484.884: [CMS-concurrent-reset-start]
> 1484.893: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1486.893: [GC [1 CMS-initial-mark: 12849K(21428K)] 109304K(139444K),
> 0.0118470 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1486.905: [CMS-concurrent-mark-start]
> 1486.921: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1486.921: [CMS-concurrent-preclean-start]
> 1486.921: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1486.921: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1491.968:
> [CMS-concurrent-abortable-preclean: 0.720/5.047 secs] [Times:
> user=0.72 sys=0.00, real=5.05 secs]
> 1491.968: [GC[YG occupancy: 96774 K (118016 K)]1491.968: [Rescan
> (parallel) , 0.0122850 secs]1491.981: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 109624K(139444K), 0.0123880 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1491.981: [CMS-concurrent-sweep-start]
> 1491.982: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1491.982: [CMS-concurrent-reset-start]
> 1491.991: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1493.991: [GC [1 CMS-initial-mark: 12849K(21428K)] 109753K(139444K),
> 0.0119790 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1494.004: [CMS-concurrent-mark-start]
> 1494.019: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1494.019: [CMS-concurrent-preclean-start]
> 1494.019: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1494.019: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1499.100:
> [CMS-concurrent-abortable-preclean: 0.722/5.080 secs] [Times:
> user=0.72 sys=0.00, real=5.08 secs]
> 1499.100: [GC[YG occupancy: 98295 K (118016 K)]1499.100: [Rescan
> (parallel) , 0.0123180 secs]1499.112: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 111145K(139444K), 0.0124240 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1499.113: [CMS-concurrent-sweep-start]
> 1499.114: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1499.114: [CMS-concurrent-reset-start]
> 1499.123: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1501.123: [GC [1 CMS-initial-mark: 12849K(21428K)] 111274K(139444K),
> 0.0117720 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]
> 1501.135: [CMS-concurrent-mark-start]
> 1501.150: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 1501.150: [CMS-concurrent-preclean-start]
> 1501.151: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.01 sys=0.00, real=0.00 secs]
> 1501.151: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1506.172:
> [CMS-concurrent-abortable-preclean: 0.712/5.022 secs] [Times:
> user=0.71 sys=0.00, real=5.02 secs]
> 1506.172: [GC[YG occupancy: 98890 K (118016 K)]1506.173: [Rescan
> (parallel) , 0.0113790 secs]1506.184: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 111740K(139444K), 0.0114830 secs]
> [Times: user=0.13 sys=0.00, real=0.02 secs]
> 1506.184: [CMS-concurrent-sweep-start]
> 1506.186: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1506.186: [CMS-concurrent-reset-start]
> 1506.195: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1508.196: [GC [1 CMS-initial-mark: 12849K(21428K)] 111868K(139444K),
> 0.0122930 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1508.208: [CMS-concurrent-mark-start]
> 1508.225: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1508.225: [CMS-concurrent-preclean-start]
> 1508.225: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1508.226: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1513.232:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1513.232: [GC[YG occupancy: 99339 K (118016 K)]1513.232: [Rescan
> (parallel) , 0.0123890 secs]1513.244: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 112189K(139444K), 0.0124930 secs]
> [Times: user=0.14 sys=0.00, real=0.02 secs]
> 1513.245: [CMS-concurrent-sweep-start]
> 1513.246: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1513.246: [CMS-concurrent-reset-start]
> 1513.255: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1515.256: [GC [1 CMS-initial-mark: 12849K(21428K)] 113182K(139444K),
> 0.0123210 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1515.268: [CMS-concurrent-mark-start]
> 1515.285: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1515.285: [CMS-concurrent-preclean-start]
> 1515.285: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1515.285: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1520.290:
> [CMS-concurrent-abortable-preclean: 0.699/5.004 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1520.290: [GC[YG occupancy: 100653 K (118016 K)]1520.290: [Rescan
> (parallel) , 0.0125490 secs]1520.303: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 113502K(139444K), 0.0126520 secs]
> [Times: user=0.14 sys=0.00, real=0.01 secs]
> 1520.303: [CMS-concurrent-sweep-start]
> 1520.304: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1520.304: [CMS-concurrent-reset-start]
> 1520.313: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1522.314: [GC [1 CMS-initial-mark: 12849K(21428K)] 113631K(139444K),
> 0.0118790 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1522.326: [CMS-concurrent-mark-start]
> 1522.343: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1522.343: [CMS-concurrent-preclean-start]
> 1522.343: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1522.343: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1527.350:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1527.350: [GC[YG occupancy: 101102 K (118016 K)]1527.350: [Rescan
> (parallel) , 0.0127460 secs]1527.363: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 113952K(139444K), 0.0128490 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1527.363: [CMS-concurrent-sweep-start]
> 1527.365: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1527.365: [CMS-concurrent-reset-start]
> 1527.374: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1529.374: [GC [1 CMS-initial-mark: 12849K(21428K)] 114080K(139444K),
> 0.0117550 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1529.386: [CMS-concurrent-mark-start]
> 1529.403: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1529.404: [CMS-concurrent-preclean-start]
> 1529.404: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1529.404: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1534.454:
> [CMS-concurrent-abortable-preclean: 0.712/5.050 secs] [Times:
> user=0.70 sys=0.01, real=5.05 secs]
> 1534.454: [GC[YG occupancy: 101591 K (118016 K)]1534.454: [Rescan
> (parallel) , 0.0122680 secs]1534.466: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 114441K(139444K), 0.0123750 secs]
> [Times: user=0.12 sys=0.02, real=0.01 secs]
> 1534.466: [CMS-concurrent-sweep-start]
> 1534.468: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1534.468: [CMS-concurrent-reset-start]
> 1534.478: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1536.478: [GC [1 CMS-initial-mark: 12849K(21428K)] 114570K(139444K),
> 0.0125250 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1536.491: [CMS-concurrent-mark-start]
> 1536.507: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1536.507: [CMS-concurrent-preclean-start]
> 1536.507: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1536.507: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1541.516:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1541.516: [GC[YG occupancy: 102041 K (118016 K)]1541.516: [Rescan
> (parallel) , 0.0088270 secs]1541.525: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 114890K(139444K), 0.0089300 secs]
> [Times: user=0.10 sys=0.00, real=0.01 secs]
> 1541.525: [CMS-concurrent-sweep-start]
> 1541.527: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1541.527: [CMS-concurrent-reset-start]
> 1541.537: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1543.537: [GC [1 CMS-initial-mark: 12849K(21428K)] 115019K(139444K),
> 0.0124500 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1543.550: [CMS-concurrent-mark-start]
> 1543.566: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1543.566: [CMS-concurrent-preclean-start]
> 1543.567: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1543.567: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1548.578:
> [CMS-concurrent-abortable-preclean: 0.706/5.011 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1548.578: [GC[YG occupancy: 102490 K (118016 K)]1548.578: [Rescan
> (parallel) , 0.0100430 secs]1548.588: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 115340K(139444K), 0.0101440 secs]
> [Times: user=0.11 sys=0.00, real=0.01 secs]
> 1548.588: [CMS-concurrent-sweep-start]
> 1548.589: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1548.589: [CMS-concurrent-reset-start]
> 1548.598: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1550.598: [GC [1 CMS-initial-mark: 12849K(21428K)] 115468K(139444K),
> 0.0125070 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1550.611: [CMS-concurrent-mark-start]
> 1550.627: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1550.627: [CMS-concurrent-preclean-start]
> 1550.628: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1550.628: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1555.631:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1555.631: [GC[YG occupancy: 103003 K (118016 K)]1555.631: [Rescan
> (parallel) , 0.0117610 secs]1555.643: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 115853K(139444K), 0.0118770 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1555.643: [CMS-concurrent-sweep-start]
> 1555.645: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1555.645: [CMS-concurrent-reset-start]
> 1555.655: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1557.655: [GC [1 CMS-initial-mark: 12849K(21428K)] 115981K(139444K),
> 0.0126720 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1557.668: [CMS-concurrent-mark-start]
> 1557.685: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1557.685: [CMS-concurrent-preclean-start]
> 1557.685: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1557.685: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1562.688:
> [CMS-concurrent-abortable-preclean: 0.697/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1562.688: [GC[YG occupancy: 103557 K (118016 K)]1562.688: [Rescan
> (parallel) , 0.0121530 secs]1562.700: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 116407K(139444K), 0.0122560 secs]
> [Times: user=0.16 sys=0.00, real=0.01 secs]
> 1562.700: [CMS-concurrent-sweep-start]
> 1562.701: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1562.701: [CMS-concurrent-reset-start]
> 1562.710: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1562.821: [GC [1 CMS-initial-mark: 12849K(21428K)] 116514K(139444K),
> 0.0127240 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1562.834: [CMS-concurrent-mark-start]
> 1562.852: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1562.852: [CMS-concurrent-preclean-start]
> 1562.853: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1562.853: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1567.859:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1567.859: [GC[YG occupancy: 104026 K (118016 K)]1567.859: [Rescan
> (parallel) , 0.0131290 secs]1567.872: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 116876K(139444K), 0.0132470 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1567.873: [CMS-concurrent-sweep-start]
> 1567.874: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1567.874: [CMS-concurrent-reset-start]
> 1567.883: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1569.883: [GC [1 CMS-initial-mark: 12849K(21428K)] 117103K(139444K),
> 0.0123770 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1569.896: [CMS-concurrent-mark-start]
> 1569.913: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1569.913: [CMS-concurrent-preclean-start]
> 1569.913: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.01 secs]
> 1569.913: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1574.920:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1574.920: [GC[YG occupancy: 104510 K (118016 K)]1574.920: [Rescan
> (parallel) , 0.0122810 secs]1574.932: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 117360K(139444K), 0.0123870 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1574.933: [CMS-concurrent-sweep-start]
> 1574.935: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1574.935: [CMS-concurrent-reset-start]
> 1574.944: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1575.163: [GC [1 CMS-initial-mark: 12849K(21428K)] 117360K(139444K),
> 0.0121590 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]
> 1575.176: [CMS-concurrent-mark-start]
> 1575.193: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1575.193: [CMS-concurrent-preclean-start]
> 1575.193: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.01 secs]
> 1575.193: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1580.197:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.71 sys=0.00, real=5.00 secs]
> 1580.197: [GC[YG occupancy: 104831 K (118016 K)]1580.197: [Rescan
> (parallel) , 0.0129860 secs]1580.210: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 117681K(139444K), 0.0130980 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1580.210: [CMS-concurrent-sweep-start]
> 1580.211: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1580.211: [CMS-concurrent-reset-start]
> 1580.220: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1582.220: [GC [1 CMS-initial-mark: 12849K(21428K)] 117809K(139444K),
> 0.0129700 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1582.234: [CMS-concurrent-mark-start]
> 1582.249: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.05
> sys=0.01, real=0.02 secs]
> 1582.249: [CMS-concurrent-preclean-start]
> 1582.249: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1582.249: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1587.262:
> [CMS-concurrent-abortable-preclean: 0.707/5.013 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1587.262: [GC[YG occupancy: 105280 K (118016 K)]1587.262: [Rescan
> (parallel) , 0.0134570 secs]1587.276: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 118130K(139444K), 0.0135720 secs]
> [Times: user=0.15 sys=0.00, real=0.02 secs]
> 1587.276: [CMS-concurrent-sweep-start]
> 1587.278: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1587.278: [CMS-concurrent-reset-start]
> 1587.287: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1589.287: [GC [1 CMS-initial-mark: 12849K(21428K)] 118258K(139444K),
> 0.0130010 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1589.301: [CMS-concurrent-mark-start]
> 1589.316: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1589.316: [CMS-concurrent-preclean-start]
> 1589.316: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1589.316: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1594.364:
> [CMS-concurrent-abortable-preclean: 0.712/5.048 secs] [Times:
> user=0.71 sys=0.00, real=5.05 secs]
> 1594.365: [GC[YG occupancy: 105770 K (118016 K)]1594.365: [Rescan
> (parallel) , 0.0131190 secs]1594.378: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 118620K(139444K), 0.0132380 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1594.378: [CMS-concurrent-sweep-start]
> 1594.380: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1594.380: [CMS-concurrent-reset-start]
> 1594.389: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1596.390: [GC [1 CMS-initial-mark: 12849K(21428K)] 118748K(139444K),
> 0.0130650 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1596.403: [CMS-concurrent-mark-start]
> 1596.418: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1596.418: [CMS-concurrent-preclean-start]
> 1596.419: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1596.419: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1601.422:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.69 sys=0.01, real=5.00 secs]
> 1601.422: [GC[YG occupancy: 106219 K (118016 K)]1601.422: [Rescan
> (parallel) , 0.0130310 secs]1601.435: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 119069K(139444K), 0.0131490 secs]
> [Times: user=0.16 sys=0.00, real=0.02 secs]
> 1601.435: [CMS-concurrent-sweep-start]
> 1601.437: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1601.437: [CMS-concurrent-reset-start]
> 1601.446: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1603.447: [GC [1 CMS-initial-mark: 12849K(21428K)] 119197K(139444K),
> 0.0130220 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1603.460: [CMS-concurrent-mark-start]
> 1603.476: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1603.476: [CMS-concurrent-preclean-start]
> 1603.476: [CMS-concurrent-preclean: 0.000/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1603.476: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1608.478:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1608.478: [GC[YG occupancy: 106668 K (118016 K)]1608.479: [Rescan
> (parallel) , 0.0122680 secs]1608.491: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 119518K(139444K), 0.0123790 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1608.491: [CMS-concurrent-sweep-start]
> 1608.492: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1608.492: [CMS-concurrent-reset-start]
> 1608.501: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1610.502: [GC [1 CMS-initial-mark: 12849K(21428K)] 119646K(139444K),
> 0.0130770 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1610.515: [CMS-concurrent-mark-start]
> 1610.530: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1610.530: [CMS-concurrent-preclean-start]
> 1610.530: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1610.530: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1615.536:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1615.536: [GC[YG occupancy: 107117 K (118016 K)]1615.536: [Rescan
> (parallel) , 0.0125470 secs]1615.549: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 119967K(139444K), 0.0126510 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1615.549: [CMS-concurrent-sweep-start]
> 1615.551: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1615.551: [CMS-concurrent-reset-start]
> 1615.561: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1617.561: [GC [1 CMS-initial-mark: 12849K(21428K)] 120095K(139444K),
> 0.0129520 secs] [Times: user=0.00 sys=0.00, real=0.02 secs]
> 1617.574: [CMS-concurrent-mark-start]
> 1617.591: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1617.591: [CMS-concurrent-preclean-start]
> 1617.591: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1617.591: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1622.598:
> [CMS-concurrent-abortable-preclean: 0.702/5.007 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1622.598: [GC[YG occupancy: 107777 K (118016 K)]1622.599: [Rescan
> (parallel) , 0.0140340 secs]1622.613: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 120627K(139444K), 0.0141520 secs]
> [Times: user=0.16 sys=0.00, real=0.01 secs]
> 1622.613: [CMS-concurrent-sweep-start]
> 1622.614: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1622.614: [CMS-concurrent-reset-start]
> 1622.623: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.02 secs]
> 1622.848: [GC [1 CMS-initial-mark: 12849K(21428K)] 120691K(139444K),
> 0.0133410 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1622.861: [CMS-concurrent-mark-start]
> 1622.878: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1622.878: [CMS-concurrent-preclean-start]
> 1622.879: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1622.879: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1627.941:
> [CMS-concurrent-abortable-preclean: 0.656/5.062 secs] [Times:
> user=0.65 sys=0.00, real=5.06 secs]
> 1627.941: [GC[YG occupancy: 108202 K (118016 K)]1627.941: [Rescan
> (parallel) , 0.0135120 secs]1627.955: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 121052K(139444K), 0.0136620 secs]
> [Times: user=0.15 sys=0.00, real=0.02 secs]
> 1627.955: [CMS-concurrent-sweep-start]
> 1627.956: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1627.956: [CMS-concurrent-reset-start]
> 1627.965: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1629.966: [GC [1 CMS-initial-mark: 12849K(21428K)] 121180K(139444K),
> 0.0133770 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1629.979: [CMS-concurrent-mark-start]
> 1629.995: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1629.995: [CMS-concurrent-preclean-start]
> 1629.996: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1629.996: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1634.998:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.69 sys=0.00, real=5.00 secs]
> 1634.999: [GC[YG occupancy: 108651 K (118016 K)]1634.999: [Rescan
> (parallel) , 0.0134300 secs]1635.012: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 121501K(139444K), 0.0135530 secs]
> [Times: user=0.16 sys=0.00, real=0.01 secs]
> 1635.012: [CMS-concurrent-sweep-start]
> 1635.014: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1635.014: [CMS-concurrent-reset-start]
> 1635.023: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1637.023: [GC [1 CMS-initial-mark: 12849K(21428K)] 121629K(139444K),
> 0.0127330 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1637.036: [CMS-concurrent-mark-start]
> 1637.053: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1637.054: [CMS-concurrent-preclean-start]
> 1637.054: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1637.054: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1642.062:
> [CMS-concurrent-abortable-preclean: 0.703/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1642.062: [GC[YG occupancy: 109100 K (118016 K)]1642.062: [Rescan
> (parallel) , 0.0124310 secs]1642.075: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 121950K(139444K), 0.0125510 secs]
> [Times: user=0.16 sys=0.00, real=0.02 secs]
> 1642.075: [CMS-concurrent-sweep-start]
> 1642.077: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1642.077: [CMS-concurrent-reset-start]
> 1642.086: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1644.087: [GC [1 CMS-initial-mark: 12849K(21428K)] 122079K(139444K),
> 0.0134300 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1644.100: [CMS-concurrent-mark-start]
> 1644.116: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1644.116: [CMS-concurrent-preclean-start]
> 1644.116: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1644.116: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1649.125:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1649.126: [GC[YG occupancy: 109549 K (118016 K)]1649.126: [Rescan
> (parallel) , 0.0126870 secs]1649.138: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 122399K(139444K), 0.0128010 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1649.139: [CMS-concurrent-sweep-start]
> 1649.141: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1649.141: [CMS-concurrent-reset-start]
> 1649.150: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1651.150: [GC [1 CMS-initial-mark: 12849K(21428K)] 122528K(139444K),
> 0.0134790 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1651.164: [CMS-concurrent-mark-start]
> 1651.179: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1651.179: [CMS-concurrent-preclean-start]
> 1651.179: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1651.179: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1656.254:
> [CMS-concurrent-abortable-preclean: 0.722/5.074 secs] [Times:
> user=0.71 sys=0.01, real=5.07 secs]
> 1656.254: [GC[YG occupancy: 110039 K (118016 K)]1656.254: [Rescan
> (parallel) , 0.0092110 secs]1656.263: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 122889K(139444K), 0.0093170 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1656.263: [CMS-concurrent-sweep-start]
> 1656.266: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1656.266: [CMS-concurrent-reset-start]
> 1656.275: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1658.275: [GC [1 CMS-initial-mark: 12849K(21428K)] 123017K(139444K),
> 0.0134150 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1658.289: [CMS-concurrent-mark-start]
> 1658.305: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1658.306: [CMS-concurrent-preclean-start]
> 1658.306: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1658.306: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1663.393:
> [CMS-concurrent-abortable-preclean: 0.711/5.087 secs] [Times:
> user=0.71 sys=0.00, real=5.08 secs]
> 1663.393: [GC[YG occupancy: 110488 K (118016 K)]1663.393: [Rescan
> (parallel) , 0.0132450 secs]1663.406: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 123338K(139444K), 0.0133600 secs]
> [Times: user=0.15 sys=0.00, real=0.02 secs]
> 1663.407: [CMS-concurrent-sweep-start]
> 1663.409: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1663.409: [CMS-concurrent-reset-start]
> 1663.418: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1665.418: [GC [1 CMS-initial-mark: 12849K(21428K)] 123467K(139444K),
> 0.0135570 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1665.432: [CMS-concurrent-mark-start]
> 1665.447: [CMS-concurrent-mark: 0.015/0.015 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1665.447: [CMS-concurrent-preclean-start]
> 1665.448: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1665.448: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1670.457:
> [CMS-concurrent-abortable-preclean: 0.704/5.009 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1670.457: [GC[YG occupancy: 110937 K (118016 K)]1670.457: [Rescan
> (parallel) , 0.0142820 secs]1670.471: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 123787K(139444K), 0.0144010 secs]
> [Times: user=0.16 sys=0.00, real=0.01 secs]
> 1670.472: [CMS-concurrent-sweep-start]
> 1670.473: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1670.473: [CMS-concurrent-reset-start]
> 1670.482: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1672.482: [GC [1 CMS-initial-mark: 12849K(21428K)] 123916K(139444K),
> 0.0136110 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1672.496: [CMS-concurrent-mark-start]
> 1672.513: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1672.513: [CMS-concurrent-preclean-start]
> 1672.513: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1672.513: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1677.530:
> [CMS-concurrent-abortable-preclean: 0.711/5.017 secs] [Times:
> user=0.71 sys=0.00, real=5.02 secs]
> 1677.530: [GC[YG occupancy: 111387 K (118016 K)]1677.530: [Rescan
> (parallel) , 0.0129210 secs]1677.543: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 124236K(139444K), 0.0130360 secs]
> [Times: user=0.16 sys=0.00, real=0.02 secs]
> 1677.543: [CMS-concurrent-sweep-start]
> 1677.545: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1677.545: [CMS-concurrent-reset-start]
> 1677.554: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1679.554: [GC [1 CMS-initial-mark: 12849K(21428K)] 124365K(139444K),
> 0.0125140 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1679.567: [CMS-concurrent-mark-start]
> 1679.584: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1679.584: [CMS-concurrent-preclean-start]
> 1679.584: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1679.584: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1684.631:
> [CMS-concurrent-abortable-preclean: 0.714/5.047 secs] [Times:
> user=0.72 sys=0.00, real=5.04 secs]
> 1684.631: [GC[YG occupancy: 112005 K (118016 K)]1684.631: [Rescan
> (parallel) , 0.0146760 secs]1684.646: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 124855K(139444K), 0.0147930 secs]
> [Times: user=0.16 sys=0.00, real=0.02 secs]
> 1684.646: [CMS-concurrent-sweep-start]
> 1684.648: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1684.648: [CMS-concurrent-reset-start]
> 1684.656: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1686.656: [GC [1 CMS-initial-mark: 12849K(21428K)] 125048K(139444K),
> 0.0138340 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1686.670: [CMS-concurrent-mark-start]
> 1686.686: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1686.686: [CMS-concurrent-preclean-start]
> 1686.687: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1686.687: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1691.689:
> [CMS-concurrent-abortable-preclean: 0.697/5.002 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1691.689: [GC[YG occupancy: 112518 K (118016 K)]1691.689: [Rescan
> (parallel) , 0.0142600 secs]1691.703: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 12849K(21428K)] 125368K(139444K), 0.0143810 secs]
> [Times: user=0.16 sys=0.00, real=0.02 secs]
> 1691.703: [CMS-concurrent-sweep-start]
> 1691.705: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1691.705: [CMS-concurrent-reset-start]
> 1691.714: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1693.714: [GC [1 CMS-initial-mark: 12849K(21428K)] 125497K(139444K),
> 0.0126710 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1693.727: [CMS-concurrent-mark-start]
> 1693.744: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1693.744: [CMS-concurrent-preclean-start]
> 1693.745: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1693.745: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1698.747:
> [CMS-concurrent-abortable-preclean: 0.698/5.003 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1698.748: [GC[YG occupancy: 112968 K (118016 K)]1698.748: [Rescan
> (parallel) , 0.0147370 secs]1698.762: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 125818K(139444K), 0.0148490 secs]
> [Times: user=0.17 sys=0.00, real=0.01 secs]
> 1698.763: [CMS-concurrent-sweep-start]
> 1698.764: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1698.764: [CMS-concurrent-reset-start]
> 1698.773: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1700.773: [GC [1 CMS-initial-mark: 12849K(21428K)] 125946K(139444K),
> 0.0128810 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1700.786: [CMS-concurrent-mark-start]
> 1700.804: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1700.804: [CMS-concurrent-preclean-start]
> 1700.804: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1700.804: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1705.810:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1705.810: [GC[YG occupancy: 113417 K (118016 K)]1705.810: [Rescan
> (parallel) , 0.0146750 secs]1705.825: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 126267K(139444K), 0.0147760 secs]
> [Times: user=0.17 sys=0.00, real=0.02 secs]
> 1705.825: [CMS-concurrent-sweep-start]
> 1705.827: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1705.827: [CMS-concurrent-reset-start]
> 1705.836: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1707.836: [GC [1 CMS-initial-mark: 12849K(21428K)] 126395K(139444K),
> 0.0137570 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1707.850: [CMS-concurrent-mark-start]
> 1707.866: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1707.866: [CMS-concurrent-preclean-start]
> 1707.867: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1707.867: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1712.878:
> [CMS-concurrent-abortable-preclean: 0.706/5.011 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1712.878: [GC[YG occupancy: 113866 K (118016 K)]1712.878: [Rescan
> (parallel) , 0.0116340 secs]1712.890: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 126716K(139444K), 0.0117350 secs]
> [Times: user=0.12 sys=0.00, real=0.01 secs]
> 1712.890: [CMS-concurrent-sweep-start]
> 1712.893: [CMS-concurrent-sweep: 0.002/0.003 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1712.893: [CMS-concurrent-reset-start]
> 1712.902: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1714.902: [GC [1 CMS-initial-mark: 12849K(21428K)] 126984K(139444K),
> 0.0134590 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1714.915: [CMS-concurrent-mark-start]
> 1714.933: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1714.933: [CMS-concurrent-preclean-start]
> 1714.934: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1714.934: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1719.940:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.71 sys=0.00, real=5.00 secs]
> 1719.940: [GC[YG occupancy: 114552 K (118016 K)]1719.940: [Rescan
> (parallel) , 0.0141320 secs]1719.955: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 127402K(139444K), 0.0142280 secs]
> [Times: user=0.16 sys=0.01, real=0.02 secs]
> 1719.955: [CMS-concurrent-sweep-start]
> 1719.956: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1719.956: [CMS-concurrent-reset-start]
> 1719.965: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1721.966: [GC [1 CMS-initial-mark: 12849K(21428K)] 127530K(139444K),
> 0.0139120 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1721.980: [CMS-concurrent-mark-start]
> 1721.996: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1721.996: [CMS-concurrent-preclean-start]
> 1721.997: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1721.997: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1727.010:
> [CMS-concurrent-abortable-preclean: 0.708/5.013 secs] [Times:
> user=0.71 sys=0.00, real=5.01 secs]
> 1727.010: [GC[YG occupancy: 115000 K (118016 K)]1727.010: [Rescan
> (parallel) , 0.0123190 secs]1727.023: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 12849K(21428K)] 127850K(139444K), 0.0124420 secs]
> [Times: user=0.15 sys=0.00, real=0.01 secs]
> 1727.023: [CMS-concurrent-sweep-start]
> 1727.024: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1727.024: [CMS-concurrent-reset-start]
> 1727.033: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1729.034: [GC [1 CMS-initial-mark: 12849K(21428K)] 127978K(139444K),
> 0.0129330 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1729.047: [CMS-concurrent-mark-start]
> 1729.064: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1729.064: [CMS-concurrent-preclean-start]
> 1729.064: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1729.064: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1734.075:
> [CMS-concurrent-abortable-preclean: 0.706/5.011 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1734.075: [GC[YG occupancy: 115449 K (118016 K)]1734.075: [Rescan
> (parallel) , 0.0131600 secs]1734.088: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 12849K(21428K)] 128298K(139444K), 0.0132810 secs]
> [Times: user=0.16 sys=0.00, real=0.01 secs]
> 1734.089: [CMS-concurrent-sweep-start]
> 1734.091: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1734.091: [CMS-concurrent-reset-start]
> 1734.100: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1736.100: [GC [1 CMS-initial-mark: 12849K(21428K)] 128427K(139444K),
> 0.0141000 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]
> 1736.115: [CMS-concurrent-mark-start]
> 1736.130: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1736.131: [CMS-concurrent-preclean-start]
> 1736.131: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1736.131: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1741.139:
> [CMS-concurrent-abortable-preclean: 0.702/5.008 secs] [Times:
> user=0.70 sys=0.00, real=5.01 secs]
> 1741.139: [GC[YG occupancy: 115897 K (118016 K)]1741.139: [Rescan
> (parallel) , 0.0146880 secs]1741.154: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 12849K(21428K)] 128747K(139444K), 0.0148020 secs]
> [Times: user=0.17 sys=0.00, real=0.02 secs]
> 1741.154: [CMS-concurrent-sweep-start]
> 1741.156: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1741.156: [CMS-concurrent-reset-start]
> 1741.165: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1742.898: [GC [1 CMS-initial-mark: 12849K(21428K)] 129085K(139444K),
> 0.0144050 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1742.913: [CMS-concurrent-mark-start]
> 1742.931: [CMS-concurrent-mark: 0.017/0.019 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1742.931: [CMS-concurrent-preclean-start]
> 1742.932: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1742.932: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1748.016:
> [CMS-concurrent-abortable-preclean: 0.728/5.084 secs] [Times:
> user=0.73 sys=0.00, real=5.09 secs]
> 1748.016: [GC[YG occupancy: 116596 K (118016 K)]1748.016: [Rescan
> (parallel) , 0.0149950 secs]1748.031: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 129446K(139444K), 0.0150970 secs]
> [Times: user=0.17 sys=0.00, real=0.01 secs]
> 1748.031: [CMS-concurrent-sweep-start]
> 1748.033: [CMS-concurrent-sweep: 0.002/0.002 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1748.033: [CMS-concurrent-reset-start]
> 1748.041: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1750.042: [GC [1 CMS-initial-mark: 12849K(21428K)] 129574K(139444K),
> 0.0141840 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1750.056: [CMS-concurrent-mark-start]
> 1750.073: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1750.073: [CMS-concurrent-preclean-start]
> 1750.074: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1750.074: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1755.080:
> [CMS-concurrent-abortable-preclean: 0.701/5.006 secs] [Times:
> user=0.70 sys=0.00, real=5.00 secs]
> 1755.080: [GC[YG occupancy: 117044 K (118016 K)]1755.080: [Rescan
> (parallel) , 0.0155560 secs]1755.096: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 12849K(21428K)] 129894K(139444K), 0.0156580 secs]
> [Times: user=0.17 sys=0.00, real=0.02 secs]
> 1755.096: [CMS-concurrent-sweep-start]
> 1755.097: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1755.097: [CMS-concurrent-reset-start]
> 1755.105: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1756.660: [GC 1756.660: [ParNew: 117108K->482K(118016K), 0.0081410
> secs] 129958K->24535K(144568K), 0.0083030 secs] [Times: user=0.05
> sys=0.01, real=0.01 secs]
> 1756.668: [GC [1 CMS-initial-mark: 24053K(26552K)] 24599K(144568K),
> 0.0015280 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1756.670: [CMS-concurrent-mark-start]
> 1756.688: [CMS-concurrent-mark: 0.016/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1756.688: [CMS-concurrent-preclean-start]
> 1756.689: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1756.689: [GC[YG occupancy: 546 K (118016 K)]1756.689: [Rescan
> (parallel) , 0.0018170 secs]1756.691: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(26552K)] 24599K(144568K), 0.0019050 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1756.691: [CMS-concurrent-sweep-start]
> 1756.694: [CMS-concurrent-sweep: 0.004/0.004 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1756.694: [CMS-concurrent-reset-start]
> 1756.704: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1758.704: [GC [1 CMS-initial-mark: 24053K(40092K)] 25372K(158108K),
> 0.0014030 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1758.705: [CMS-concurrent-mark-start]
> 1758.720: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.05
> sys=0.00, real=0.01 secs]
> 1758.720: [CMS-concurrent-preclean-start]
> 1758.720: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.01 sys=0.00, real=0.00 secs]
> 1758.721: [GC[YG occupancy: 1319 K (118016 K)]1758.721: [Rescan
> (parallel) , 0.0014940 secs]1758.722: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 25372K(158108K), 0.0015850 secs]
> [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1758.722: [CMS-concurrent-sweep-start]
> 1758.726: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1758.726: [CMS-concurrent-reset-start]
> 1758.735: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1760.735: [GC [1 CMS-initial-mark: 24053K(40092K)] 25565K(158108K),
> 0.0014530 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1760.737: [CMS-concurrent-mark-start]
> 1760.755: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1760.755: [CMS-concurrent-preclean-start]
> 1760.755: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1760.756: [GC[YG occupancy: 1512 K (118016 K)]1760.756: [Rescan
> (parallel) , 0.0014970 secs]1760.757: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 25565K(158108K), 0.0015980 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1760.757: [CMS-concurrent-sweep-start]
> 1760.761: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1760.761: [CMS-concurrent-reset-start]
> 1760.770: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1762.770: [GC [1 CMS-initial-mark: 24053K(40092K)] 25693K(158108K),
> 0.0013680 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1762.772: [CMS-concurrent-mark-start]
> 1762.788: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1762.788: [CMS-concurrent-preclean-start]
> 1762.788: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1762.788: [GC[YG occupancy: 1640 K (118016 K)]1762.789: [Rescan
> (parallel) , 0.0020360 secs]1762.791: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 25693K(158108K), 0.0021450 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1762.791: [CMS-concurrent-sweep-start]
> 1762.794: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1762.794: [CMS-concurrent-reset-start]
> 1762.803: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1764.804: [GC [1 CMS-initial-mark: 24053K(40092K)] 26747K(158108K),
> 0.0014620 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1764.805: [CMS-concurrent-mark-start]
> 1764.819: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1764.819: [CMS-concurrent-preclean-start]
> 1764.820: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1764.820: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1769.835:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 1769.835: [GC[YG occupancy: 3015 K (118016 K)]1769.835: [Rescan
> (parallel) , 0.0010360 secs]1769.836: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 27068K(158108K), 0.0011310 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1769.837: [CMS-concurrent-sweep-start]
> 1769.840: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1769.840: [CMS-concurrent-reset-start]
> 1769.849: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1771.850: [GC [1 CMS-initial-mark: 24053K(40092K)] 27196K(158108K),
> 0.0014740 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1771.851: [CMS-concurrent-mark-start]
> 1771.868: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1771.868: [CMS-concurrent-preclean-start]
> 1771.868: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1771.868: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1776.913:
> [CMS-concurrent-abortable-preclean: 0.112/5.044 secs] [Times:
> user=0.12 sys=0.00, real=5.04 secs]
> 1776.913: [GC[YG occupancy: 4052 K (118016 K)]1776.913: [Rescan
> (parallel) , 0.0017790 secs]1776.915: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 28105K(158108K), 0.0018790 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1776.915: [CMS-concurrent-sweep-start]
> 1776.918: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1776.918: [CMS-concurrent-reset-start]
> 1776.927: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1778.928: [GC [1 CMS-initial-mark: 24053K(40092K)] 28233K(158108K),
> 0.0015470 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1778.929: [CMS-concurrent-mark-start]
> 1778.947: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1778.947: [CMS-concurrent-preclean-start]
> 1778.947: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1778.947: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1783.963:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1783.963: [GC[YG occupancy: 4505 K (118016 K)]1783.963: [Rescan
> (parallel) , 0.0014480 secs]1783.965: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 28558K(158108K), 0.0015470 secs]
> [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1783.965: [CMS-concurrent-sweep-start]
> 1783.968: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1783.968: [CMS-concurrent-reset-start]
> 1783.977: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1785.978: [GC [1 CMS-initial-mark: 24053K(40092K)] 28686K(158108K),
> 0.0015760 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1785.979: [CMS-concurrent-mark-start]
> 1785.996: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1785.996: [CMS-concurrent-preclean-start]
> 1785.996: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1785.996: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1791.009:
> [CMS-concurrent-abortable-preclean: 0.107/5.013 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1791.010: [GC[YG occupancy: 4954 K (118016 K)]1791.010: [Rescan
> (parallel) , 0.0020030 secs]1791.012: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 29007K(158108K), 0.0021040 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1791.012: [CMS-concurrent-sweep-start]
> 1791.015: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1791.015: [CMS-concurrent-reset-start]
> 1791.023: [CMS-concurrent-reset: 0.008/0.008 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1793.023: [GC [1 CMS-initial-mark: 24053K(40092K)] 29136K(158108K),
> 0.0017200 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1793.025: [CMS-concurrent-mark-start]
> 1793.044: [CMS-concurrent-mark: 0.019/0.019 secs] [Times: user=0.08
> sys=0.00, real=0.02 secs]
> 1793.044: [CMS-concurrent-preclean-start]
> 1793.045: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1793.045: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1798.137:
> [CMS-concurrent-abortable-preclean: 0.112/5.093 secs] [Times:
> user=0.11 sys=0.00, real=5.09 secs]
> 1798.137: [GC[YG occupancy: 6539 K (118016 K)]1798.137: [Rescan
> (parallel) , 0.0016650 secs]1798.139: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 30592K(158108K), 0.0017600 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1798.139: [CMS-concurrent-sweep-start]
> 1798.143: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1798.143: [CMS-concurrent-reset-start]
> 1798.152: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1800.152: [GC [1 CMS-initial-mark: 24053K(40092K)] 30721K(158108K),
> 0.0016650 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1800.154: [CMS-concurrent-mark-start]
> 1800.170: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1800.170: [CMS-concurrent-preclean-start]
> 1800.171: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1800.171: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1805.181:
> [CMS-concurrent-abortable-preclean: 0.110/5.010 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 1805.181: [GC[YG occupancy: 8090 K (118016 K)]1805.181: [Rescan
> (parallel) , 0.0018850 secs]1805.183: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 32143K(158108K), 0.0019860 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1805.183: [CMS-concurrent-sweep-start]
> 1805.187: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1805.187: [CMS-concurrent-reset-start]
> 1805.196: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1807.196: [GC [1 CMS-initial-mark: 24053K(40092K)] 32272K(158108K),
> 0.0018760 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1807.198: [CMS-concurrent-mark-start]
> 1807.216: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1807.216: [CMS-concurrent-preclean-start]
> 1807.216: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1807.216: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1812.232:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1812.232: [GC[YG occupancy: 8543 K (118016 K)]1812.232: [Rescan
> (parallel) , 0.0020890 secs]1812.234: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 32596K(158108K), 0.0021910 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1812.234: [CMS-concurrent-sweep-start]
> 1812.238: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1812.238: [CMS-concurrent-reset-start]
> 1812.247: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1812.928: [GC [1 CMS-initial-mark: 24053K(40092K)] 32661K(158108K),
> 0.0019710 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1812.930: [CMS-concurrent-mark-start]
> 1812.947: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1812.947: [CMS-concurrent-preclean-start]
> 1812.947: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1812.948: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1817.963:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1817.963: [GC[YG occupancy: 8928 K (118016 K)]1817.963: [Rescan
> (parallel) , 0.0011790 secs]1817.964: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 32981K(158108K), 0.0012750 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1817.964: [CMS-concurrent-sweep-start]
> 1817.968: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1817.968: [CMS-concurrent-reset-start]
> 1817.977: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1819.977: [GC [1 CMS-initial-mark: 24053K(40092K)] 33110K(158108K),
> 0.0018900 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1819.979: [CMS-concurrent-mark-start]
> 1819.996: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1819.997: [CMS-concurrent-preclean-start]
> 1819.997: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1819.997: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1825.012:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1825.013: [GC[YG occupancy: 9377 K (118016 K)]1825.013: [Rescan
> (parallel) , 0.0020580 secs]1825.015: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 33431K(158108K), 0.0021510 secs]
> [Times: user=0.01 sys=0.00, real=0.01 secs]
> 1825.015: [CMS-concurrent-sweep-start]
> 1825.018: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1825.018: [CMS-concurrent-reset-start]
> 1825.027: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1827.028: [GC [1 CMS-initial-mark: 24053K(40092K)] 33559K(158108K),
> 0.0019140 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1827.030: [CMS-concurrent-mark-start]
> 1827.047: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1827.047: [CMS-concurrent-preclean-start]
> 1827.047: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1827.047: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1832.066:
> [CMS-concurrent-abortable-preclean: 0.109/5.018 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1832.066: [GC[YG occupancy: 9827 K (118016 K)]1832.066: [Rescan
> (parallel) , 0.0019440 secs]1832.068: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 33880K(158108K), 0.0020410 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1832.068: [CMS-concurrent-sweep-start]
> 1832.071: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1832.071: [CMS-concurrent-reset-start]
> 1832.081: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1832.935: [GC [1 CMS-initial-mark: 24053K(40092K)] 34093K(158108K),
> 0.0019830 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1832.937: [CMS-concurrent-mark-start]
> 1832.954: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1832.954: [CMS-concurrent-preclean-start]
> 1832.955: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1832.955: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1837.970:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1837.970: [GC[YG occupancy: 10349 K (118016 K)]1837.970: [Rescan
> (parallel) , 0.0019670 secs]1837.972: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 34402K(158108K), 0.0020800 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1837.972: [CMS-concurrent-sweep-start]
> 1837.976: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1837.976: [CMS-concurrent-reset-start]
> 1837.985: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1839.985: [GC [1 CMS-initial-mark: 24053K(40092K)] 34531K(158108K),
> 0.0020220 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1839.987: [CMS-concurrent-mark-start]
> 1840.005: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.06
> sys=0.01, real=0.02 secs]
> 1840.005: [CMS-concurrent-preclean-start]
> 1840.006: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1840.006: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1845.018:
> [CMS-concurrent-abortable-preclean: 0.106/5.012 secs] [Times:
> user=0.10 sys=0.01, real=5.01 secs]
> 1845.018: [GC[YG occupancy: 10798 K (118016 K)]1845.018: [Rescan
> (parallel) , 0.0015500 secs]1845.019: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 34851K(158108K), 0.0016500 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1845.020: [CMS-concurrent-sweep-start]
> 1845.023: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1845.023: [CMS-concurrent-reset-start]
> 1845.032: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1847.032: [GC [1 CMS-initial-mark: 24053K(40092K)] 34980K(158108K),
> 0.0020600 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1847.035: [CMS-concurrent-mark-start]
> 1847.051: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.01 secs]
> 1847.051: [CMS-concurrent-preclean-start]
> 1847.052: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1847.052: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1852.067:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 1852.067: [GC[YG occupancy: 11247 K (118016 K)]1852.067: [Rescan
> (parallel) , 0.0011880 secs]1852.069: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 35300K(158108K), 0.0012900 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1852.069: [CMS-concurrent-sweep-start]
> 1852.072: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1852.072: [CMS-concurrent-reset-start]
> 1852.081: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1854.082: [GC [1 CMS-initial-mark: 24053K(40092K)] 35429K(158108K),
> 0.0021010 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1854.084: [CMS-concurrent-mark-start]
> 1854.100: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1854.100: [CMS-concurrent-preclean-start]
> 1854.101: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1854.101: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1859.116:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1859.116: [GC[YG occupancy: 11701 K (118016 K)]1859.117: [Rescan
> (parallel) , 0.0010230 secs]1859.118: [weak refs processing, 0.0000130
> secs] [1 CMS-remark: 24053K(40092K)] 35754K(158108K), 0.0011230 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1859.118: [CMS-concurrent-sweep-start]
> 1859.121: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1859.121: [CMS-concurrent-reset-start]
> 1859.130: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1861.131: [GC [1 CMS-initial-mark: 24053K(40092K)] 35882K(158108K),
> 0.0021240 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1861.133: [CMS-concurrent-mark-start]
> 1861.149: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1861.149: [CMS-concurrent-preclean-start]
> 1861.150: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1861.150: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1866.220:
> [CMS-concurrent-abortable-preclean: 0.112/5.070 secs] [Times:
> user=0.12 sys=0.00, real=5.07 secs]
> 1866.220: [GC[YG occupancy: 12388 K (118016 K)]1866.220: [Rescan
> (parallel) , 0.0027090 secs]1866.223: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 36441K(158108K), 0.0028070 secs]
> [Times: user=0.02 sys=0.00, real=0.01 secs]
> 1866.223: [CMS-concurrent-sweep-start]
> 1866.227: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1866.227: [CMS-concurrent-reset-start]
> 1866.236: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1868.236: [GC [1 CMS-initial-mark: 24053K(40092K)] 36569K(158108K),
> 0.0023650 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1868.239: [CMS-concurrent-mark-start]
> 1868.256: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1868.256: [CMS-concurrent-preclean-start]
> 1868.257: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1868.257: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1873.267:
> [CMS-concurrent-abortable-preclean: 0.105/5.011 secs] [Times:
> user=0.13 sys=0.00, real=5.01 secs]
> 1873.268: [GC[YG occupancy: 12837 K (118016 K)]1873.268: [Rescan
> (parallel) , 0.0018720 secs]1873.270: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 36890K(158108K), 0.0019730 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1873.270: [CMS-concurrent-sweep-start]
> 1873.273: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1873.273: [CMS-concurrent-reset-start]
> 1873.282: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1875.283: [GC [1 CMS-initial-mark: 24053K(40092K)] 37018K(158108K),
> 0.0024410 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1875.285: [CMS-concurrent-mark-start]
> 1875.302: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1875.302: [CMS-concurrent-preclean-start]
> 1875.302: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1875.303: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1880.318:
> [CMS-concurrent-abortable-preclean: 0.110/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1880.318: [GC[YG occupancy: 13286 K (118016 K)]1880.318: [Rescan
> (parallel) , 0.0023860 secs]1880.321: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 37339K(158108K), 0.0024910 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1880.321: [CMS-concurrent-sweep-start]
> 1880.324: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1880.324: [CMS-concurrent-reset-start]
> 1880.333: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 1882.334: [GC [1 CMS-initial-mark: 24053K(40092K)] 37467K(158108K),
> 0.0024090 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1882.336: [CMS-concurrent-mark-start]
> 1882.352: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1882.352: [CMS-concurrent-preclean-start]
> 1882.353: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1882.353: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1887.368:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1887.368: [GC[YG occupancy: 13739 K (118016 K)]1887.368: [Rescan
> (parallel) , 0.0022370 secs]1887.370: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 37792K(158108K), 0.0023360 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1887.371: [CMS-concurrent-sweep-start]
> 1887.374: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1887.374: [CMS-concurrent-reset-start]
> 1887.383: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1889.384: [GC [1 CMS-initial-mark: 24053K(40092K)] 37920K(158108K),
> 0.0024690 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1889.386: [CMS-concurrent-mark-start]
> 1889.404: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1889.404: [CMS-concurrent-preclean-start]
> 1889.405: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.01 sys=0.00, real=0.00 secs]
> 1889.405: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1894.488:
> [CMS-concurrent-abortable-preclean: 0.112/5.083 secs] [Times:
> user=0.11 sys=0.00, real=5.08 secs]
> 1894.488: [GC[YG occupancy: 14241 K (118016 K)]1894.488: [Rescan
> (parallel) , 0.0020670 secs]1894.490: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 38294K(158108K), 0.0021630 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1894.490: [CMS-concurrent-sweep-start]
> 1894.494: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1894.494: [CMS-concurrent-reset-start]
> 1894.503: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1896.503: [GC [1 CMS-initial-mark: 24053K(40092K)] 38422K(158108K),
> 0.0025430 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1896.506: [CMS-concurrent-mark-start]
> 1896.524: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1896.524: [CMS-concurrent-preclean-start]
> 1896.525: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1896.525: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1901.540:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1901.540: [GC[YG occupancy: 14690 K (118016 K)]1901.540: [Rescan
> (parallel) , 0.0014810 secs]1901.542: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 38743K(158108K), 0.0015820 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1901.542: [CMS-concurrent-sweep-start]
> 1901.545: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1901.545: [CMS-concurrent-reset-start]
> 1901.555: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1903.555: [GC [1 CMS-initial-mark: 24053K(40092K)] 38871K(158108K),
> 0.0025990 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1903.558: [CMS-concurrent-mark-start]
> 1903.575: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1903.575: [CMS-concurrent-preclean-start]
> 1903.576: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1903.576: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1908.586:
> [CMS-concurrent-abortable-preclean: 0.105/5.010 secs] [Times:
> user=0.10 sys=0.00, real=5.01 secs]
> 1908.587: [GC[YG occupancy: 15207 K (118016 K)]1908.587: [Rescan
> (parallel) , 0.0026240 secs]1908.589: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 39260K(158108K), 0.0027260 secs]
> [Times: user=0.01 sys=0.00, real=0.00 secs]
> 1908.589: [CMS-concurrent-sweep-start]
> 1908.593: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1908.593: [CMS-concurrent-reset-start]
> 1908.602: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1910.602: [GC [1 CMS-initial-mark: 24053K(40092K)] 39324K(158108K),
> 0.0025610 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1910.605: [CMS-concurrent-mark-start]
> 1910.621: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1910.621: [CMS-concurrent-preclean-start]
> 1910.622: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.01 sys=0.00, real=0.00 secs]
> 1910.622: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1915.684:
> [CMS-concurrent-abortable-preclean: 0.112/5.062 secs] [Times:
> user=0.11 sys=0.00, real=5.07 secs]
> 1915.684: [GC[YG occupancy: 15592 K (118016 K)]1915.684: [Rescan
> (parallel) , 0.0023940 secs]1915.687: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 39645K(158108K), 0.0025050 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1915.687: [CMS-concurrent-sweep-start]
> 1915.690: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1915.690: [CMS-concurrent-reset-start]
> 1915.699: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1917.700: [GC [1 CMS-initial-mark: 24053K(40092K)] 39838K(158108K),
> 0.0025010 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1917.702: [CMS-concurrent-mark-start]
> 1917.719: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1917.719: [CMS-concurrent-preclean-start]
> 1917.719: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1917.719: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1922.735:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.01, real=5.02 secs]
> 1922.735: [GC[YG occupancy: 16198 K (118016 K)]1922.735: [Rescan
> (parallel) , 0.0028750 secs]1922.738: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 40251K(158108K), 0.0029760 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1922.738: [CMS-concurrent-sweep-start]
> 1922.741: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1922.741: [CMS-concurrent-reset-start]
> 1922.751: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1922.957: [GC [1 CMS-initial-mark: 24053K(40092K)] 40324K(158108K),
> 0.0027380 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1922.960: [CMS-concurrent-mark-start]
> 1922.978: [CMS-concurrent-mark: 0.017/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1922.978: [CMS-concurrent-preclean-start]
> 1922.979: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1922.979: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1927.994:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 1927.995: [GC[YG occupancy: 16645 K (118016 K)]1927.995: [Rescan
> (parallel) , 0.0013210 secs]1927.996: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 40698K(158108K), 0.0017610 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1927.996: [CMS-concurrent-sweep-start]
> 1928.000: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1928.000: [CMS-concurrent-reset-start]
> 1928.009: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1930.009: [GC [1 CMS-initial-mark: 24053K(40092K)] 40826K(158108K),
> 0.0028310 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1930.012: [CMS-concurrent-mark-start]
> 1930.028: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1930.028: [CMS-concurrent-preclean-start]
> 1930.029: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1930.029: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1935.044:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1935.045: [GC[YG occupancy: 17098 K (118016 K)]1935.045: [Rescan
> (parallel) , 0.0015440 secs]1935.046: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 41151K(158108K), 0.0016490 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1935.046: [CMS-concurrent-sweep-start]
> 1935.050: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1935.050: [CMS-concurrent-reset-start]
> 1935.059: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1937.059: [GC [1 CMS-initial-mark: 24053K(40092K)] 41279K(158108K),
> 0.0028290 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1937.062: [CMS-concurrent-mark-start]
> 1937.079: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1937.079: [CMS-concurrent-preclean-start]
> 1937.079: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1937.079: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1942.095:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.01, real=5.02 secs]
> 1942.095: [GC[YG occupancy: 17547 K (118016 K)]1942.095: [Rescan
> (parallel) , 0.0030270 secs]1942.098: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 41600K(158108K), 0.0031250 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1942.098: [CMS-concurrent-sweep-start]
> 1942.101: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1942.101: [CMS-concurrent-reset-start]
> 1942.111: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1944.111: [GC [1 CMS-initial-mark: 24053K(40092K)] 41728K(158108K),
> 0.0028080 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1944.114: [CMS-concurrent-mark-start]
> 1944.130: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1944.130: [CMS-concurrent-preclean-start]
> 1944.131: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1944.131: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1949.146:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1949.146: [GC[YG occupancy: 17996 K (118016 K)]1949.146: [Rescan
> (parallel) , 0.0028800 secs]1949.149: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 42049K(158108K), 0.0029810 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1949.149: [CMS-concurrent-sweep-start]
> 1949.152: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1949.152: [CMS-concurrent-reset-start]
> 1949.162: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1951.162: [GC [1 CMS-initial-mark: 24053K(40092K)] 42177K(158108K),
> 0.0028760 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1951.165: [CMS-concurrent-mark-start]
> 1951.184: [CMS-concurrent-mark: 0.019/0.019 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1951.184: [CMS-concurrent-preclean-start]
> 1951.184: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1951.184: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1956.244:
> [CMS-concurrent-abortable-preclean: 0.112/5.059 secs] [Times:
> user=0.11 sys=0.01, real=5.05 secs]
> 1956.244: [GC[YG occupancy: 18498 K (118016 K)]1956.244: [Rescan
> (parallel) , 0.0019760 secs]1956.246: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 42551K(158108K), 0.0020750 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 1956.246: [CMS-concurrent-sweep-start]
> 1956.249: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1956.249: [CMS-concurrent-reset-start]
> 1956.259: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1958.259: [GC [1 CMS-initial-mark: 24053K(40092K)] 42747K(158108K),
> 0.0029160 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1958.262: [CMS-concurrent-mark-start]
> 1958.279: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1958.279: [CMS-concurrent-preclean-start]
> 1958.279: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1958.279: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1963.295:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 1963.295: [GC[YG occupancy: 18951 K (118016 K)]1963.295: [Rescan
> (parallel) , 0.0020140 secs]1963.297: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 43004K(158108K), 0.0021100 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1963.297: [CMS-concurrent-sweep-start]
> 1963.300: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1963.300: [CMS-concurrent-reset-start]
> 1963.310: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1965.310: [GC [1 CMS-initial-mark: 24053K(40092K)] 43132K(158108K),
> 0.0029420 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1965.313: [CMS-concurrent-mark-start]
> 1965.329: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1965.329: [CMS-concurrent-preclean-start]
> 1965.330: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1965.330: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1970.345:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 1970.345: [GC[YG occupancy: 19400 K (118016 K)]1970.345: [Rescan
> (parallel) , 0.0031610 secs]1970.349: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 43453K(158108K), 0.0032580 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1970.349: [CMS-concurrent-sweep-start]
> 1970.352: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1970.352: [CMS-concurrent-reset-start]
> 1970.361: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1972.362: [GC [1 CMS-initial-mark: 24053K(40092K)] 43581K(158108K),
> 0.0029960 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1972.365: [CMS-concurrent-mark-start]
> 1972.381: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 1972.381: [CMS-concurrent-preclean-start]
> 1972.382: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1972.382: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1977.397:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1977.398: [GC[YG occupancy: 19849 K (118016 K)]1977.398: [Rescan
> (parallel) , 0.0018110 secs]1977.399: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 43902K(158108K), 0.0019100 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1977.400: [CMS-concurrent-sweep-start]
> 1977.403: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1977.403: [CMS-concurrent-reset-start]
> 1977.412: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1979.413: [GC [1 CMS-initial-mark: 24053K(40092K)] 44031K(158108K),
> 0.0030240 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 1979.416: [CMS-concurrent-mark-start]
> 1979.434: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.08
> sys=0.00, real=0.02 secs]
> 1979.434: [CMS-concurrent-preclean-start]
> 1979.434: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1979.434: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1984.511:
> [CMS-concurrent-abortable-preclean: 0.112/5.077 secs] [Times:
> user=0.12 sys=0.00, real=5.07 secs]
> 1984.511: [GC[YG occupancy: 20556 K (118016 K)]1984.511: [Rescan
> (parallel) , 0.0032740 secs]1984.514: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 44609K(158108K), 0.0033720 secs]
> [Times: user=0.03 sys=0.00, real=0.01 secs]
> 1984.515: [CMS-concurrent-sweep-start]
> 1984.518: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1984.518: [CMS-concurrent-reset-start]
> 1984.527: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1986.528: [GC [1 CMS-initial-mark: 24053K(40092K)] 44737K(158108K),
> 0.0032890 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1986.531: [CMS-concurrent-mark-start]
> 1986.548: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 1986.548: [CMS-concurrent-preclean-start]
> 1986.548: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1986.548: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1991.564:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 1991.564: [GC[YG occupancy: 21005 K (118016 K)]1991.564: [Rescan
> (parallel) , 0.0022540 secs]1991.566: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 45058K(158108K), 0.0023650 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 1991.566: [CMS-concurrent-sweep-start]
> 1991.570: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 1991.570: [CMS-concurrent-reset-start]
> 1991.579: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 1993.579: [GC [1 CMS-initial-mark: 24053K(40092K)] 45187K(158108K),
> 0.0032480 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 1993.583: [CMS-concurrent-mark-start]
> 1993.599: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 1993.599: [CMS-concurrent-preclean-start]
> 1993.600: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 1993.600: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 1998.688:
> [CMS-concurrent-abortable-preclean: 0.112/5.089 secs] [Times:
> user=0.10 sys=0.01, real=5.09 secs]
> 1998.689: [GC[YG occupancy: 21454 K (118016 K)]1998.689: [Rescan
> (parallel) , 0.0025510 secs]1998.691: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 45507K(158108K), 0.0026500 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 1998.691: [CMS-concurrent-sweep-start]
> 1998.695: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 1998.695: [CMS-concurrent-reset-start]
> 1998.704: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 2000.704: [GC [1 CMS-initial-mark: 24053K(40092K)] 45636K(158108K),
> 0.0033350 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2000.708: [CMS-concurrent-mark-start]
> 2000.726: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2000.726: [CMS-concurrent-preclean-start]
> 2000.726: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2000.726: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2005.742:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 2005.742: [GC[YG occupancy: 21968 K (118016 K)]2005.742: [Rescan
> (parallel) , 0.0027300 secs]2005.745: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 46021K(158108K), 0.0028560 secs]
> [Times: user=0.02 sys=0.01, real=0.01 secs]
> 2005.745: [CMS-concurrent-sweep-start]
> 2005.748: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2005.748: [CMS-concurrent-reset-start]
> 2005.757: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.01, real=0.01 secs]
> 2007.758: [GC [1 CMS-initial-mark: 24053K(40092K)] 46217K(158108K),
> 0.0033290 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2007.761: [CMS-concurrent-mark-start]
> 2007.778: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2007.778: [CMS-concurrent-preclean-start]
> 2007.778: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2007.778: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2012.794:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 2012.794: [GC[YG occupancy: 22421 K (118016 K)]2012.794: [Rescan
> (parallel) , 0.0036890 secs]2012.798: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 46474K(158108K), 0.0037910 secs]
> [Times: user=0.02 sys=0.01, real=0.00 secs]
> 2012.798: [CMS-concurrent-sweep-start]
> 2012.801: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2012.801: [CMS-concurrent-reset-start]
> 2012.810: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2012.980: [GC [1 CMS-initial-mark: 24053K(40092K)] 46547K(158108K),
> 0.0033990 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 2012.984: [CMS-concurrent-mark-start]
> 2013.004: [CMS-concurrent-mark: 0.019/0.020 secs] [Times: user=0.06
> sys=0.01, real=0.02 secs]
> 2013.004: [CMS-concurrent-preclean-start]
> 2013.005: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2013.005: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2018.020:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 2018.020: [GC[YG occupancy: 22867 K (118016 K)]2018.020: [Rescan
> (parallel) , 0.0025180 secs]2018.023: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 46920K(158108K), 0.0026190 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 2018.023: [CMS-concurrent-sweep-start]
> 2018.026: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2018.026: [CMS-concurrent-reset-start]
> 2018.036: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2020.036: [GC [1 CMS-initial-mark: 24053K(40092K)] 47048K(158108K),
> 0.0034020 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2020.039: [CMS-concurrent-mark-start]
> 2020.057: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2020.057: [CMS-concurrent-preclean-start]
> 2020.058: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2020.058: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2025.073:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2025.073: [GC[YG occupancy: 23316 K (118016 K)]2025.073: [Rescan
> (parallel) , 0.0020110 secs]2025.075: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 47369K(158108K), 0.0021080 secs]
> [Times: user=0.02 sys=0.00, real=0.00 secs]
> 2025.075: [CMS-concurrent-sweep-start]
> 2025.079: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2025.079: [CMS-concurrent-reset-start]
> 2025.088: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2027.088: [GC [1 CMS-initial-mark: 24053K(40092K)] 47498K(158108K),
> 0.0034100 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2027.092: [CMS-concurrent-mark-start]
> 2027.108: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2027.108: [CMS-concurrent-preclean-start]
> 2027.109: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2027.109: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2032.120:
> [CMS-concurrent-abortable-preclean: 0.105/5.011 secs] [Times:
> user=0.10 sys=0.00, real=5.01 secs]
> 2032.120: [GC[YG occupancy: 23765 K (118016 K)]2032.120: [Rescan
> (parallel) , 0.0025970 secs]2032.123: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 47818K(158108K), 0.0026940 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 2032.123: [CMS-concurrent-sweep-start]
> 2032.126: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2032.126: [CMS-concurrent-reset-start]
> 2032.135: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2034.136: [GC [1 CMS-initial-mark: 24053K(40092K)] 47951K(158108K),
> 0.0034720 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2034.139: [CMS-concurrent-mark-start]
> 2034.156: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2034.156: [CMS-concurrent-preclean-start]
> 2034.156: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2034.156: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2039.171:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2039.172: [GC[YG occupancy: 24218 K (118016 K)]2039.172: [Rescan
> (parallel) , 0.0038590 secs]2039.176: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 48271K(158108K), 0.0039560 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2039.176: [CMS-concurrent-sweep-start]
> 2039.179: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2039.179: [CMS-concurrent-reset-start]
> 2039.188: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2041.188: [GC [1 CMS-initial-mark: 24053K(40092K)] 48400K(158108K),
> 0.0035110 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2041.192: [CMS-concurrent-mark-start]
> 2041.209: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2041.209: [CMS-concurrent-preclean-start]
> 2041.209: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2041.209: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2046.268:
> [CMS-concurrent-abortable-preclean: 0.108/5.058 secs] [Times:
> user=0.12 sys=0.00, real=5.06 secs]
> 2046.268: [GC[YG occupancy: 24813 K (118016 K)]2046.268: [Rescan
> (parallel) , 0.0042050 secs]2046.272: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 48866K(158108K), 0.0043070 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2046.272: [CMS-concurrent-sweep-start]
> 2046.275: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2046.275: [CMS-concurrent-reset-start]
> 2046.285: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2048.285: [GC [1 CMS-initial-mark: 24053K(40092K)] 48994K(158108K),
> 0.0037700 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2048.289: [CMS-concurrent-mark-start]
> 2048.307: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2048.307: [CMS-concurrent-preclean-start]
> 2048.307: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2048.307: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2053.323:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2053.323: [GC[YG occupancy: 25262 K (118016 K)]2053.323: [Rescan
> (parallel) , 0.0030780 secs]2053.326: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 49315K(158108K), 0.0031760 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2053.326: [CMS-concurrent-sweep-start]
> 2053.329: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2053.329: [CMS-concurrent-reset-start]
> 2053.338: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2055.339: [GC [1 CMS-initial-mark: 24053K(40092K)] 49444K(158108K),
> 0.0037730 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2055.343: [CMS-concurrent-mark-start]
> 2055.359: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2055.359: [CMS-concurrent-preclean-start]
> 2055.360: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2055.360: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2060.373:
> [CMS-concurrent-abortable-preclean: 0.107/5.013 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2060.373: [GC[YG occupancy: 25715 K (118016 K)]2060.373: [Rescan
> (parallel) , 0.0037090 secs]2060.377: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 49768K(158108K), 0.0038110 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2060.377: [CMS-concurrent-sweep-start]
> 2060.380: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2060.380: [CMS-concurrent-reset-start]
> 2060.389: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2062.390: [GC [1 CMS-initial-mark: 24053K(40092K)] 49897K(158108K),
> 0.0037860 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2062.394: [CMS-concurrent-mark-start]
> 2062.410: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2062.410: [CMS-concurrent-preclean-start]
> 2062.411: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2062.411: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2067.426:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 2067.427: [GC[YG occupancy: 26231 K (118016 K)]2067.427: [Rescan
> (parallel) , 0.0031980 secs]2067.430: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 50284K(158108K), 0.0033100 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2067.430: [CMS-concurrent-sweep-start]
> 2067.433: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2067.433: [CMS-concurrent-reset-start]
> 2067.443: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2069.443: [GC [1 CMS-initial-mark: 24053K(40092K)] 50412K(158108K),
> 0.0038060 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 2069.447: [CMS-concurrent-mark-start]
> 2069.465: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2069.465: [CMS-concurrent-preclean-start]
> 2069.465: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2069.465: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2074.535:
> [CMS-concurrent-abortable-preclean: 0.112/5.070 secs] [Times:
> user=0.12 sys=0.00, real=5.06 secs]
> 2074.535: [GC[YG occupancy: 26749 K (118016 K)]2074.535: [Rescan
> (parallel) , 0.0040450 secs]2074.539: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 50802K(158108K), 0.0041460 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2074.539: [CMS-concurrent-sweep-start]
> 2074.543: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2074.543: [CMS-concurrent-reset-start]
> 2074.552: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2076.552: [GC [1 CMS-initial-mark: 24053K(40092K)] 50930K(158108K),
> 0.0038960 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 2076.556: [CMS-concurrent-mark-start]
> 2076.575: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2076.575: [CMS-concurrent-preclean-start]
> 2076.575: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2076.575: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2081.590:
> [CMS-concurrent-abortable-preclean: 0.109/5.014 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2081.590: [GC[YG occupancy: 27198 K (118016 K)]2081.590: [Rescan
> (parallel) , 0.0042420 secs]2081.594: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 51251K(158108K), 0.0043450 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2081.594: [CMS-concurrent-sweep-start]
> 2081.597: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2081.597: [CMS-concurrent-reset-start]
> 2081.607: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2083.607: [GC [1 CMS-initial-mark: 24053K(40092K)] 51447K(158108K),
> 0.0038630 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2083.611: [CMS-concurrent-mark-start]
> 2083.628: [CMS-concurrent-mark: 0.017/0.017 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2083.628: [CMS-concurrent-preclean-start]
> 2083.628: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2083.628: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2088.642:
> [CMS-concurrent-abortable-preclean: 0.108/5.014 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2088.642: [GC[YG occupancy: 27651 K (118016 K)]2088.642: [Rescan
> (parallel) , 0.0031520 secs]2088.645: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 51704K(158108K), 0.0032520 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2088.645: [CMS-concurrent-sweep-start]
> 2088.649: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2088.649: [CMS-concurrent-reset-start]
> 2088.658: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2090.658: [GC [1 CMS-initial-mark: 24053K(40092K)] 51832K(158108K),
> 0.0039130 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2090.662: [CMS-concurrent-mark-start]
> 2090.678: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2090.678: [CMS-concurrent-preclean-start]
> 2090.679: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2090.679: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2095.690:
> [CMS-concurrent-abortable-preclean: 0.105/5.011 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2095.690: [GC[YG occupancy: 28100 K (118016 K)]2095.690: [Rescan
> (parallel) , 0.0024460 secs]2095.693: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 52153K(158108K), 0.0025460 secs]
> [Times: user=0.03 sys=0.00, real=0.00 secs]
> 2095.693: [CMS-concurrent-sweep-start]
> 2095.696: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2095.696: [CMS-concurrent-reset-start]
> 2095.705: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2096.616: [GC [1 CMS-initial-mark: 24053K(40092K)] 53289K(158108K),
> 0.0039340 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2096.620: [CMS-concurrent-mark-start]
> 2096.637: [CMS-concurrent-mark: 0.016/0.017 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2096.637: [CMS-concurrent-preclean-start]
> 2096.638: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2096.638: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2101.654:
> [CMS-concurrent-abortable-preclean: 0.110/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 2101.654: [GC[YG occupancy: 29557 K (118016 K)]2101.654: [Rescan
> (parallel) , 0.0034020 secs]2101.657: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 53610K(158108K), 0.0035000 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2101.657: [CMS-concurrent-sweep-start]
> 2101.661: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2101.661: [CMS-concurrent-reset-start]
> 2101.670: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2103.004: [GC [1 CMS-initial-mark: 24053K(40092K)] 53997K(158108K),
> 0.0042590 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2103.009: [CMS-concurrent-mark-start]
> 2103.027: [CMS-concurrent-mark: 0.017/0.019 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2103.027: [CMS-concurrent-preclean-start]
> 2103.028: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2103.028: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2108.043:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.10 sys=0.01, real=5.02 secs]
> 2108.043: [GC[YG occupancy: 30385 K (118016 K)]2108.044: [Rescan
> (parallel) , 0.0048950 secs]2108.048: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 54438K(158108K), 0.0049930 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2108.049: [CMS-concurrent-sweep-start]
> 2108.052: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2108.052: [CMS-concurrent-reset-start]
> 2108.061: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2110.062: [GC [1 CMS-initial-mark: 24053K(40092K)] 54502K(158108K),
> 0.0042120 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
> 2110.066: [CMS-concurrent-mark-start]
> 2110.084: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2110.084: [CMS-concurrent-preclean-start]
> 2110.085: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2110.085: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2115.100:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2115.101: [GC[YG occupancy: 30770 K (118016 K)]2115.101: [Rescan
> (parallel) , 0.0049040 secs]2115.106: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 54823K(158108K), 0.0050080 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2115.106: [CMS-concurrent-sweep-start]
> 2115.109: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2115.109: [CMS-concurrent-reset-start]
> 2115.118: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2117.118: [GC [1 CMS-initial-mark: 24053K(40092K)] 54952K(158108K),
> 0.0042490 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2117.123: [CMS-concurrent-mark-start]
> 2117.139: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2117.139: [CMS-concurrent-preclean-start]
> 2117.140: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2117.140: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2122.155:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.02 secs]
> 2122.155: [GC[YG occupancy: 31219 K (118016 K)]2122.155: [Rescan
> (parallel) , 0.0036460 secs]2122.159: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 55272K(158108K), 0.0037440 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2122.159: [CMS-concurrent-sweep-start]
> 2122.162: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2122.162: [CMS-concurrent-reset-start]
> 2122.172: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2124.172: [GC [1 CMS-initial-mark: 24053K(40092K)] 55401K(158108K),
> 0.0043010 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 2124.176: [CMS-concurrent-mark-start]
> 2124.195: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2124.195: [CMS-concurrent-preclean-start]
> 2124.195: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2124.195: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2129.211:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 2129.211: [GC[YG occupancy: 31669 K (118016 K)]2129.211: [Rescan
> (parallel) , 0.0049870 secs]2129.216: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 55722K(158108K), 0.0050860 secs]
> [Times: user=0.04 sys=0.00, real=0.01 secs]
> 2129.216: [CMS-concurrent-sweep-start]
> 2129.219: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 2129.219: [CMS-concurrent-reset-start]
> 2129.228: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2131.229: [GC [1 CMS-initial-mark: 24053K(40092K)] 55850K(158108K),
> 0.0042340 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2131.233: [CMS-concurrent-mark-start]
> 2131.249: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2131.249: [CMS-concurrent-preclean-start]
> 2131.249: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2131.249: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2136.292:
> [CMS-concurrent-abortable-preclean: 0.108/5.042 secs] [Times:
> user=0.11 sys=0.00, real=5.04 secs]
> 2136.292: [GC[YG occupancy: 32174 K (118016 K)]2136.292: [Rescan
> (parallel) , 0.0037250 secs]2136.296: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 56227K(158108K), 0.0038250 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 2136.296: [CMS-concurrent-sweep-start]
> 2136.299: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2136.299: [CMS-concurrent-reset-start]
> 2136.308: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2138.309: [GC [1 CMS-initial-mark: 24053K(40092K)] 56356K(158108K),
> 0.0043040 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2138.313: [CMS-concurrent-mark-start]
> 2138.329: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.05
> sys=0.01, real=0.02 secs]
> 2138.329: [CMS-concurrent-preclean-start]
> 2138.329: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2138.329: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2143.341:
> [CMS-concurrent-abortable-preclean: 0.106/5.011 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2143.341: [GC[YG occupancy: 32623 K (118016 K)]2143.341: [Rescan
> (parallel) , 0.0038660 secs]2143.345: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 56676K(158108K), 0.0039760 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 2143.345: [CMS-concurrent-sweep-start]
> 2143.349: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2143.349: [CMS-concurrent-reset-start]
> 2143.358: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2145.358: [GC [1 CMS-initial-mark: 24053K(40092K)] 56805K(158108K),
> 0.0043390 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2145.362: [CMS-concurrent-mark-start]
> 2145.379: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2145.379: [CMS-concurrent-preclean-start]
> 2145.379: [CMS-concurrent-preclean: 0.000/0.000 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2145.379: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2150.393:
> [CMS-concurrent-abortable-preclean: 0.108/5.014 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2150.393: [GC[YG occupancy: 33073 K (118016 K)]2150.393: [Rescan
> (parallel) , 0.0038190 secs]2150.397: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 57126K(158108K), 0.0039210 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 2150.397: [CMS-concurrent-sweep-start]
> 2150.400: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2150.400: [CMS-concurrent-reset-start]
> 2150.410: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2152.410: [GC [1 CMS-initial-mark: 24053K(40092K)] 57254K(158108K),
> 0.0044080 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2152.415: [CMS-concurrent-mark-start]
> 2152.431: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2152.431: [CMS-concurrent-preclean-start]
> 2152.432: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2152.432: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2157.447:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.01, real=5.02 secs]
> 2157.447: [GC[YG occupancy: 33522 K (118016 K)]2157.447: [Rescan
> (parallel) , 0.0038130 secs]2157.451: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 57575K(158108K), 0.0039160 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2157.451: [CMS-concurrent-sweep-start]
> 2157.454: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2157.454: [CMS-concurrent-reset-start]
> 2157.464: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 2159.464: [GC [1 CMS-initial-mark: 24053K(40092K)] 57707K(158108K),
> 0.0045170 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2159.469: [CMS-concurrent-mark-start]
> 2159.483: [CMS-concurrent-mark: 0.014/0.014 secs] [Times: user=0.06
> sys=0.00, real=0.01 secs]
> 2159.483: [CMS-concurrent-preclean-start]
> 2159.483: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2159.483: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2164.491:
> [CMS-concurrent-abortable-preclean: 0.111/5.007 secs] [Times:
> user=0.12 sys=0.00, real=5.01 secs]
> 2164.491: [GC[YG occupancy: 34293 K (118016 K)]2164.491: [Rescan
> (parallel) , 0.0052070 secs]2164.496: [weak refs processing, 0.0000120
> secs] [1 CMS-remark: 24053K(40092K)] 58347K(158108K), 0.0053130 secs]
> [Times: user=0.06 sys=0.00, real=0.01 secs]
> 2164.496: [CMS-concurrent-sweep-start]
> 2164.500: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2164.500: [CMS-concurrent-reset-start]
> 2164.509: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.01, real=0.01 secs]
> 2166.509: [GC [1 CMS-initial-mark: 24053K(40092K)] 58475K(158108K),
> 0.0045900 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2166.514: [CMS-concurrent-mark-start]
> 2166.533: [CMS-concurrent-mark: 0.019/0.019 secs] [Times: user=0.07
> sys=0.00, real=0.02 secs]
> 2166.533: [CMS-concurrent-preclean-start]
> 2166.533: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2166.533: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2171.549:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.02 secs]
> 2171.549: [GC[YG occupancy: 34743 K (118016 K)]2171.549: [Rescan
> (parallel) , 0.0052200 secs]2171.554: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 58796K(158108K), 0.0053210 secs]
> [Times: user=0.05 sys=0.00, real=0.01 secs]
> 2171.554: [CMS-concurrent-sweep-start]
> 2171.558: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2171.558: [CMS-concurrent-reset-start]
> 2171.567: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2173.567: [GC [1 CMS-initial-mark: 24053K(40092K)] 58924K(158108K),
> 0.0046700 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
> 2173.572: [CMS-concurrent-mark-start]
> 2173.588: [CMS-concurrent-mark: 0.016/0.016 secs] [Times: user=0.06
> sys=0.00, real=0.02 secs]
> 2173.588: [CMS-concurrent-preclean-start]
> 2173.589: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2173.589: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2178.604:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.10 sys=0.01, real=5.02 secs]
> 2178.605: [GC[YG occupancy: 35192 K (118016 K)]2178.605: [Rescan
> (parallel) , 0.0041460 secs]2178.609: [weak refs processing, 0.0000110
> secs] [1 CMS-remark: 24053K(40092K)] 59245K(158108K), 0.0042450 secs]
> [Times: user=0.04 sys=0.00, real=0.00 secs]
> 2178.609: [CMS-concurrent-sweep-start]
> 2178.612: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.01
> sys=0.00, real=0.00 secs]
> 2178.612: [CMS-concurrent-reset-start]
> 2178.622: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.00
> sys=0.00, real=0.01 secs]
> 2180.622: [GC [1 CMS-initial-mark: 24053K(40092K)] 59373K(158108K),
> 0.0047200 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
> 2180.627: [CMS-concurrent-mark-start]
> 2180.645: [CMS-concurrent-mark: 0.018/0.018 secs] [Times: user=0.08
> sys=0.00, real=0.02 secs]
> 2180.645: [CMS-concurrent-preclean-start]
> 2180.645: [CMS-concurrent-preclean: 0.001/0.001 secs] [Times:
> user=0.00 sys=0.00, real=0.00 secs]
> 2180.645: [CMS-concurrent-abortable-preclean-start]
>   CMS: abort preclean due to time 2185.661:
> [CMS-concurrent-abortable-preclean: 0.109/5.015 secs] [Times:
> user=0.11 sys=0.00, real=5.01 secs]
> 2185.661: [GC[YG occupancy: 35645 K (118016 K)]2185.661: [Rescan
> (parallel) , 0.0050730 secs]2185.666: [weak refs processing, 0.0000100
> secs] [1 CMS-remark: 24053K(40092K)] 59698K(158108K), 0.0051720 secs]
> [Times: user=0.04 sys=0.01, real=0.01 secs]
> 2185.666: [CMS-concurrent-sweep-start]
> 2185.670: [CMS-concurrent-sweep: 0.003/0.003 secs] [Times: user=0.00
> sys=0.00, real=0.00 secs]
> 2185.670: [CMS-concurrent-reset-start]
> 2185.679: [CMS-concurrent-reset: 0.009/0.009 secs] [Times: user=0.01
> sys=0.00, real=0.01 secs]
> 2187.679: [GC [1 CMS-initial-mark: 24053K(40092K)] 59826K(158108K),
> 0.0047350 secs]
>
> --
> gregross:)
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

-- 
Marcos Ortiz Valmaseda,
Data Engineer && Senior System Administrator at UCI
Blog: http://marcosluis2186.posterous.com
Linkedin: http://www.linkedin.com/in/marcosluis2186
Twitter: @marcosluis2186





10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

Mime
View raw message