accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACCUMULO-1113) consistency check failure on !METADATA table
Date Tue, 26 Feb 2013 20:16:15 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Newton updated ACCUMULO-1113:
----------------------------------

    Description: 
On nodeA:

{noformat}
2013-02-26 19:44:49,184 [tabletserver.MinorCompactor] DEBUG: Begin minor compaction /accumulo/tables/!0/root_tablet/F000221z.rf_tmp
!0;!0<<
2013-02-26 19:44:49,207 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MinC [memory] ->
/root_tablet/F000221z.rf
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Unassigning !0;!0<<@(null,10.0.0.8:9997[33cf2816b97c139],null)
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
2013-02-26 19:44:49,267 [state.ZooStore] DEBUG: Removing /root_tablet/location
{noformat}

On nodeB, the consistency check fails just a few seconds later:

{noformat}
2013-02-26 19:44:49,283 [tabletserver.TabletServer] INFO : Loading tablet !0;!0<<
2013-02-26 19:44:49,306 [tabletserver.TabletServer] INFO : Root tablet loaded: !0;!0<<

2013-02-26 19:45:18,254 [tabletserver.Tablet] DEBUG: Starting MajC !0;!0<< (NORMAL)
[/root_tablet/A00021zc.rf, /root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf_tmp 
[]
2013-02-26 19:45:18,371 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MajC [/root_tablet/A00021zc.rf,
/root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf

2013-02-26 19:45:21,005 [tabletserver.Tablet] ERROR: Data file in !METADATA differ from in
memory data !0;!0<<  
[
/root_tablet/A0002529.rf, 
/root_tablet/F00025hv.rf, 
/root_tablet/F000263t.rf
]  [
/root_tablet/A00021zc.rf, 
/root_tablet/F000221z.rf, 
/root_tablet/F00025hv.rf, 
/root_tablet/F000263t.rf
]
{noformat}

The unload/load between the servers was clean.  Location states were properly made in zookeeper.



  was:
On nodeA:

{noformat}
2013-02-26 19:44:49,184 [tabletserver.MinorCompactor] DEBUG: Begin minor compaction /accumulo/tables/!0/root_tablet/F000221z.rf_tmp
!0;!0<<
2013-02-26 19:44:49,207 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MinC [memory] ->
/root_tablet/F000221z.rf
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Unassigning !0;!0<<@(null,10.0.0.8:9997[33cf2816b97c139],null)
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
2013-02-26 19:44:49,267 [state.ZooStore] DEBUG: Removing /root_tablet/location
{noformat}

On nodeB, the consistency check fails just 3 seconds later:

{noformat}
2013-02-26 19:44:49,283 [tabletserver.TabletServer] INFO : Loading tablet !0;!0<<
2013-02-26 19:44:49,306 [tabletserver.TabletServer] INFO : Root tablet loaded: !0;!0<<

2013-02-26 19:45:18,254 [tabletserver.Tablet] DEBUG: Starting MajC !0;!0<< (NORMAL)
[/root_tablet/A00021zc.rf, /root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf_tmp 
[]
2013-02-26 19:45:18,371 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MajC [/root_tablet/A00021zc.rf,
/root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf

2013-02-26 19:45:21,005 [tabletserver.Tablet] ERROR: Data file in !METADATA differ from in
memory data !0;!0<<  
[
/root_tablet/A0002529.rf, 
/root_tablet/F00025hv.rf, 
/root_tablet/F000263t.rf
]  [
/root_tablet/A00021zc.rf, 
/root_tablet/F000221z.rf, 
/root_tablet/F00025hv.rf, 
/root_tablet/F000263t.rf
]
{noformat}

The unload/load between the servers was clean.  Location states were properly made in zookeeper.



    
> consistency check failure on !METADATA table
> --------------------------------------------
>
>                 Key: ACCUMULO-1113
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1113
>             Project: Accumulo
>          Issue Type: Bug
>          Components: master, tserver
>         Environment: 10 node test cluster running random walk test
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>            Priority: Critical
>              Labels: 15_qa_bug
>             Fix For: 1.5.0
>
>
> On nodeA:
> {noformat}
> 2013-02-26 19:44:49,184 [tabletserver.MinorCompactor] DEBUG: Begin minor compaction /accumulo/tables/!0/root_tablet/F000221z.rf_tmp
!0;!0<<
> 2013-02-26 19:44:49,207 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MinC [memory]
-> /root_tablet/F000221z.rf
> 2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Unassigning !0;!0<<@(null,10.0.0.8:9997[33cf2816b97c139],null)
> 2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
> 2013-02-26 19:44:49,267 [tabletserver.TabletServer] DEBUG: Failed to unload tablet !0;!0<<...
it was alread closing or closed : Tablet !0;!0<< already closed closing closeComplete
> 2013-02-26 19:44:49,267 [state.ZooStore] DEBUG: Removing /root_tablet/location
> {noformat}
> On nodeB, the consistency check fails just a few seconds later:
> {noformat}
> 2013-02-26 19:44:49,283 [tabletserver.TabletServer] INFO : Loading tablet !0;!0<<
> 2013-02-26 19:44:49,306 [tabletserver.TabletServer] INFO : Root tablet loaded: !0;!0<<
> 2013-02-26 19:45:18,254 [tabletserver.Tablet] DEBUG: Starting MajC !0;!0<< (NORMAL)
[/root_tablet/A00021zc.rf, /root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf_tmp 
[]
> 2013-02-26 19:45:18,371 [tabletserver.Tablet] TABLET_HIST: !0;!0<< MajC [/root_tablet/A00021zc.rf,
/root_tablet/F000221z.rf] --> /root_tablet/A0002529.rf
> 2013-02-26 19:45:21,005 [tabletserver.Tablet] ERROR: Data file in !METADATA differ from
in memory data !0;!0<<  
> [
> /root_tablet/A0002529.rf, 
> /root_tablet/F00025hv.rf, 
> /root_tablet/F000263t.rf
> ]  [
> /root_tablet/A00021zc.rf, 
> /root_tablet/F000221z.rf, 
> /root_tablet/F00025hv.rf, 
> /root_tablet/F000263t.rf
> ]
> {noformat}
> The unload/load between the servers was clean.  Location states were properly made in
zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message