accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1773) Garbage collector may delete referenced files after upgrade
Date Thu, 24 Oct 2013 16:12:01 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804348#comment-13804348
] 

Keith Turner commented on ACCUMULO-1773:
----------------------------------------

bq. Don't we need fully qualified URIs to handle multiple namenodes anyway?

Any new file refs stored in the metadata table in 1.6.0 will use full qualified URIs.  However
relative paths can still exist in metadata from earlier versions.  

Unrelated to your question, the following situation is one that I was thinking of.  Excluding
relative paths, there can still be problems if configurations differ spatially and/or temporally
  

 # Tserver A writes a ref to metadata for hdfs://foo.com:123/accumulo/tables/9/default_tablet/0.rf
 # Later Tserver B writes a del marker to metadata for hdfs://foo:123/accumulo/tables/9/default_tablet/0.rf
 (its configured differently than Tserver A)
 # Accumulo GC deletes hdfs://foo:123/accumulo/tables/9/default_tablet/0.rf

In the situation above a file thats referenced was deleted.  The problem is that foo:123 !=
foo.com:123 even though they resolve to the same IP.   If the Accumulo GC used /tables/9/default_tablet/0.rf
instead of the fully qualified URIs, then it would not delete the file.  




> Garbage collector may delete referenced files after upgrade
> -----------------------------------------------------------
>
>                 Key: ACCUMULO-1773
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1773
>             Project: Accumulo
>          Issue Type: Sub-task
>          Components: master, tserver
>            Reporter: Keith Turner
>            Assignee: Keith Turner
>            Priority: Blocker
>             Fix For: 1.6.0
>
>
> Looking at the srouce code, it seems like the garbage collector uses a mixture of relative
and absolute paths when determining what files to delete.  I think if a deletion candidate
is an absolute path and the reference is a relative path then it could delete the referenced
file.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message