hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy" <sa...@pearsonwholesale.com>
Subject hbase split error
Date Sun, 18 Nov 2007 01:56:25 GMT
I running release 0.15.0

I have one table that is trying to split on a restart of hbase but it keep 
failing and exiting after a not a valid DFS filename error

this is from the regionserver log:

2007-11-17 15:18:45,542 FATAL org.apache.hadoop.hbase.HRegionServer: Set 
stop flag in regionserver/0:0:0:0:0:0:0:0:60020.splitOrCompactChecker
java.lang.IllegalArgumentException: Pathname 
/gfs_storage/hadoop-root/hbase/hregion_webdata,,-3862545529986602998/splits/hregion_webdata,com.tripod.beifaust/robots.txt:http,7515331023182364597

from 
/gfs_storage/hadoop-root/hbase/hregion_webdata,,-3862545529986602998/splits/hregion_webdata,com.tripod.beifaust/robots.txt:http,7515331023182364597

is not a valid DFS filename.
        at 
org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:102)
        at 
org.apache.hadoop.dfs.DistributedFileSystem.exists(DistributedFileSystem.java:147)
        at org.apache.hadoop.hbase.HRegion.closeAndSplit(HRegion.java:428)
        at 
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.split(HRegionServer.java:217)
        at 
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.checkForSplitsOrCompactions(HRegionServer.java:202)
        at 
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.chore(HRegionServer.java:186)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:58)



after the error the regionserver closes all IPC Server handler's and exits

I thank the problem is what I am testing with is spider pages form the web I 
store the pages under rows like google does in bigtable

http://www.example.com/
becomes
com.example.www/:http

That's my row key looking at the error looks like a row key with a / in it 
kills the split because the path name is broken by the /'s from the row url 
key.

Any suggestions on a fix besides changing my row key?
I could change my key to change the /'s to \'s but I thank this problem 
might come up later on some how the / in the key need to be escaped or 
something
do I need to open up a bug report for this?

Billy




Mime
View raw message