hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1274) TestMergeTable is broken in Hudson
Date Sun, 22 Mar 2009 01:23:50 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688091#action_12688091

Nitay Joffe commented on HBASE-1274:

Ah, thanks for the info Stack, that helps explain a lot.

The test used to pass because the tableName.toString() would not match anything and no merge
would actually occur. Now, we actually try to merge and fail.
The test does not actually assert anything, which is why it was passing blindly. There should
always be two regions after the merge, so I can just add check for that, right?

bq. There is already an issue to deal with the case where there is a sequenceid clash; unlikely
but a possibility.

I think the issue you mention, HBASE-1212, is actually the cause here.

We initially create three regions using this:

     HRegion[] regions = {
       createAregion(null, row_70001, 1, 70000),
       createAregion(row_70001, row_80001, 70001, 10000),
       createAregion(row_80001, null, 80001, 10000)

The last two create store files with the same sequence id.

bq. The fact that sequenceids are now clashing was probably always there but before hfile,
we'd just move on one of the sequenceids

I don't think we actually currently do the logic you describe. Where is that code?

bq. a suspect practise and something we no longer can do now the meta is kept in hfile...
can't rewrite

I noticed that we use random filenames. What about using something that sorts lexicographically
and sorting the dir.listFiles we use when grabbing the store files? Then when we move around
two files with the same sequence ids to do a merge we can rename the actual files.

bq. so perhaps the thing to do if can't figure anything else is to make the TestMergeTable
make irregular files with diffrening numbers of edits.

I can make that change, it's very easy. Should I just do that and leave the merging of same
sequenceid files for another issue?

> TestMergeTable is broken in Hudson
> ----------------------------------
>                 Key: HBASE-1274
>                 URL: https://issues.apache.org/jira/browse/HBASE-1274
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: test
>            Reporter: Nitay Joffe
>            Assignee: Nitay Joffe
>            Priority: Minor
>             Fix For: 0.20.0
> http://hudson.zones.apache.org/hudson/job/HBase-Patch/539/testReport/org.apache.hadoop.hbase/TestMergeTable/testMergeTable/
> java.io.IOException: Files have same sequenceid
> 	at org.apache.hadoop.hbase.regionserver.HRegion.merge(HRegion.java:2500)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.mergeAdjacent(HRegion.java:2412)
> 	at org.apache.hadoop.hbase.HMerge$Merger.merge(HMerge.java:167)
> 	at org.apache.hadoop.hbase.HMerge$Merger.process(HMerge.java:126)
> 	at org.apache.hadoop.hbase.HMerge.merge(HMerge.java:91)
> 	at org.apache.hadoop.hbase.TestMergeTable.testMergeTable(TestMergeTable.java:35)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message