hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clint Morgan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2056) A table with row keys containing colon fails to split regions
Date Fri, 19 Oct 2007 16:44:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536260
] 

Clint Morgan commented on HADOOP-2056:
--------------------------------------

Splitting still fails for me with key names containing colon and at-sign:

2007-10-19 09:05:40,817 INFO org.apache.hadoop.hbase.HRegion: Splitting WebResource,,1192808631244
because largest aggrega
te size is 98.5m and desired size is 64.0m
2007-10-19 09:05:42,396 INFO org.apache.hadoop.hbase.HRegion: closed WebResource,,1192808631244
2007-10-19 09:05:42,536 INFO org.apache.hadoop.hbase.HRegion: region WebResource,,1192809940817
available
2007-10-19 09:05:42,540 ERROR org.apache.hadoop.hbase.HRegionServer: Split or compaction failed
java.io.IOException: file:/opt/benchmark/clint/hbase/hregion_WebResource,,1192808631244/splits/hregion_WebResource,O5KaNMK
gS3GjQL4dQZ0mDaWoS6-uAmxYNHtpNaBiP5xpRq_iOmtrSrRjQr0_Qa_iOmxbOLt_Ra4g%0AMq_iOawiP6Gh,1192809940817/C:
No such file or dire
ctory
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:134)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:116)
        at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:180)
        at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:377)
        at org.apache.hadoop.hbase.HRegion.<init>(HRegion.java:264)
        at org.apache.hadoop.hbase.HRegion.closeAndSplit(HRegion.java:479)
        at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.split(HRegionServer.java:219)
        at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.checkForSplitsOrCompactions(HRegionServer.java:204)
        at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.chore(HRegionServer.java:188)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:58)

Looking at the log and my splits directory, I think it is caused by the % in '..._Ra4g%0AMq...'.


Examining with ls, shows a '?' instead of the percentage:

ls hbase/hregion_WebResource,,1192808631244/splits/
hregion_WebResource,O5KaNMKgS3GjQL4dQZ0mDaWoS6-uAmxYNHtpNaBiP5xpRq_iOmtrSrRjQr0_Qa_iOmxbOLt_Ra4g?Mq_iOawiP6Gh,1192809940817

And 'ls --show-control-chars' has a carriage return in the file name at that location

ls --show-control-chars  hbase/hregion_WebResource,,11
hregion_WebResource,O5KaNMKgS3GjQL4dQZ0mDaWoS6-uAmxYNHtpNaBiP5xpRq_iOmtrSrRjQr0_Qa_iOmxbOLt_Ra4g
Mq_iOawiP6Gh,1192809940817


> A table with row keys containing colon fails to split regions
> -------------------------------------------------------------
>
>                 Key: HADOOP-2056
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2056
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>    Affects Versions: 0.16.0
>         Environment: latest trunk of both hadoop and hbase
>            Reporter: Johan Oskarsson
>            Assignee: Jim Kellerman
>             Fix For: 0.16.0
>
>         Attachments: patch.txt
>
>
> I assume it's my dodgy key names that triggered this. Replaced the colons with dots and
then this exception wasn't thrown anymore.
> 2007-10-15 13:59:34,732 INFO org.apache.hadoop.hbase.HRegion: Splitting 
> charts-overall,,-5504750682370618723 because largest aggregate size is 
> 103.4m and desired size is 64.0m
> 2007-10-15 13:59:34,772 INFO org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker:

> regionserver/0:0:0:0:0:0:0:0:60020.splitOrCompactChecker exiting 
> 2007-10-15 13:59:34,774 FATAL org.apache.hadoop.hbase.HRegionServer: Set stop flag in
regionserver/0:0:0:0:0:0:0:0:60020.splitOrCompactChecker
> java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: hregion_charts-overall,charts:5522565:overall,6424564033243158021
>         at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>         at org.apache.hadoop.fs.Path.<init>(Path.java:126)
>         at org.apache.hadoop.hbase.HRegion.getRegionDir(HRegion.java:1872)
>         at org.apache.hadoop.hbase.HRegion.getSplitRegionDir(HRegion.java:503)
>         at org.apache.hadoop.hbase.HRegion.closeAndSplit(HRegion.java:427)
>         at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.split(HRegionServer.java:217)
>         at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.checkForSplitsOrCompactions(HRegionServer.java:202)
>         at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.chore(HRegionServer.java:186)
>         at org.apache.hadoop.hbase.Chore.run(Chore.java:58)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> hregion_charts-overall,charts:5522565:overall,6424564033243158021
>         at java.net.URI.checkPath(URI.java:1787)
>         at java.net.URI.<init>(URI.java:735)
>         at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>         ... 8 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message