hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HBASE-1092) FileNotFoundException trying to get index file size for metrics
Date Thu, 25 Jun 2009 22:52:07 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack resolved HBASE-1092.
--------------------------

    Resolution: Cannot Reproduce

Haven't seen since.  Resolving.

> FileNotFoundException trying to get index file size for metrics
> ---------------------------------------------------------------
>
>                 Key: HBASE-1092
>                 URL: https://issues.apache.org/jira/browse/HBASE-1092
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: apurtell cluster; trunk on hadoop 0.18.1.
>            Reporter: stack
>            Assignee: stack
>
> This is an odd one.  We open a region and load up its store files.  Part of loading store
files is confirming presence of MapFile index files (reconstituting them even if missing).
 The below log is of region open and then seconds later, failing to find the index files when
we go to look at them for sake of metrics:
> {code}
> 2008-12-28 00:06:19,330 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_OPEN: content,10a1c144cf729885001e71a5ff5108dc,1230416158498
> 2008-12-28 00:06:19,330 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region
content,10a1c144cf729885001e71a5ff5108dc,1230416158498/2030495720
> 2008-12-28 00:06:19,337 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/4139998553412763261,
isReference=false, sequence id=13310628, length=54275, majorCompaction=false
> 2008-12-28 00:06:19,368 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/4467064002967944271,
isReference=false, sequence id=9760762, length=432827, majorCompaction=false
> 2008-12-28 00:06:19,373 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/5563124412728188459,
isReference=false, sequence id=12406630, length=22596, majorCompaction=false
> 2008-12-28 00:06:19,379 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/7040855870376599550,
isReference=false, sequence id=12795530, length=4163, majorCompaction=false
> 2008-12-28 00:06:19,379 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s)
in hstore 2030495720/url, max sequence id 13310628
> 2008-12-28 00:06:19,496 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0,
skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:19,687 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/4076087643455354411,
isReference=false, sequence id=9760762, length=1491212, majorCompaction=false
> 2008-12-28 00:06:19,691 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/4178444212265859440,
isReference=false, sequence id=13310628, length=156148, majorCompaction=false
> 2008-12-28 00:06:19,697 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/7223110203696352566,
isReference=false, sequence id=12406630, length=84614, majorCompaction=false
> 2008-12-28 00:06:19,703 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/8629305049543986640,
isReference=false, sequence id=12795530, length=9293, majorCompaction=false
> 2008-12-28 00:06:19,704 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s)
in hstore 2030495720/info, max sequence id 13310628
> 2008-12-28 00:06:19,773 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/content/mapfiles/3159110292991346789
to /data/hbase/content/888098363/content/mapfiles/6086595812879433437
> 2008-12-28 00:06:19,787 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed
 compaction of 888098363/content store size is 172.4m
> 2008-12-28 00:06:19,791 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction
size of 888098363/info: 1.4m; Skipped 2 file(s), size: 1221094
> 2008-12-28 00:06:19,801 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction
of 2 file(s)  into /data/hbase/content/compaction.dir/888098363/info/mapfiles/2317624608256855622
> 2008-12-28 00:06:19,821 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0,
skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:19,879 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/info/mapfiles/2317624608256855622
to /data/hbase/content/888098363/info/mapfiles/2738033219360665217
> 2008-12-28 00:06:19,883 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/2253403598042963153,
isReference=false, sequence id=13310628, length=12491312, majorCompaction=false
> 2008-12-28 00:06:19,887 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/4596284995168293361,
isReference=false, sequence id=9760762, length=158138153, majorCompaction=false
> 2008-12-28 00:06:19,896 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed
 compaction of 888098363/info store size is 1.4m
> 2008-12-28 00:06:19,898 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction
size of 888098363/url: 405.9k; Skipped 2 file(s), size: 339984
> 2008-12-28 00:06:19,904 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction
of 2 file(s)  into /data/hbase/content/compaction.dir/888098363/url/mapfiles/1108952767537472093
> 2008-12-28 00:06:19,924 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/7464468628166271570,
isReference=false, sequence id=12406630, length=11058492, majorCompaction=false
> 2008-12-28 00:06:19,930 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/7703689073557380324,
isReference=false, sequence id=12795530, length=183695, majorCompaction=false
> 2008-12-28 00:06:19,931 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s)
in hstore 2030495720/content, max sequence id 13310628
> 2008-12-28 00:06:19,958 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/url/mapfiles/1108952767537472093
to /data/hbase/content/888098363/url/mapfiles/4113677425818108069
> 2008-12-28 00:06:19,976 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed
 compaction of 888098363/url store size is 403.6k
> 2008-12-28 00:06:19,978 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
completed on region content,846510f382e9a0ae5655f03cb772830d,1230384248550 in 0sec
> 2008-12-28 00:06:19,979 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting 
compaction on region content,e2f10daf46269ad3cc25766aa3bf48c4,1230416165744
> 2008-12-28 00:06:19,981 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction
size of 2100409777/content: 170.2m; Skipped 1 file(s), size: 144757669
> 2008-12-28 00:06:19,996 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction
of 2 file(s)  into /data/hbase/content/compaction.dir/2100409777/content/mapfiles/7166480659109830957
> 2008-12-28 00:06:20,033 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0,
skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:20,127 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Deleting
old log file: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/oldlogfile.log
> 2008-12-28 00:06:20,137 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence
id for region content,10a1c144cf729885001e71a5ff5108dc,1230416158498 is 13310629
> 2008-12-28 00:06:20,138 INFO org.apache.hadoop.hbase.regionserver.HRegion: region content,10a1c144cf729885001e71a5ff5108dc,1230416158498/2030495720
available
> ....
> 2008-12-28 00:06:21,008 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction
of 2 file(s)  into /data/hbase/content/compaction.dir/2100409777/url/mapfiles/744229794307693860
> 2008-12-28 00:06:21,030 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error
getting store file index size for 2030495720/content: java.io.FileNotFoundException: File
does not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/content/mapfiles/7703689073557380324/index
>         at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
>         at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
>         at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)
>         at java.lang.Thread.run(Thread.java:619)
> 2008-12-28 00:06:21,032 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error
getting store file index size for 2030495720/info: java.io.FileNotFoundException: File does
not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/info/mapfiles/8629305049543986640/index
>         at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
>         at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
>         at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)

>         at java.lang.Thread.run(Thread.java:619)
> 2008-12-28 00:06:21,053 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error
getting store file index size for 2030495720/url: java.io.FileNotFoundException: File does
not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/url/mapfiles/7040855870376599550/index
>         at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
>         at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
>         at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)
>         at java.lang.Thread.run(Thread.java:619)
> ...
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message