hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore
Date Fri, 20 Apr 2018 23:04:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446485#comment-16446485
] 

Aaron Fabbri commented on HADOOP-13649:
---------------------------------------

Assigning to [~gabor.bota].  Feel free to change the datastructures used. The "two hashtables"
was a quick and dirty design that may not be the best if we add TTL-based expiry.  I would
look into [~stevel@apache.org]'s suggestion to use Guava cache as well.  Not sure if it has
the needed functionality to support prune and all the other operations that go with it (note
that prune() sometimes needs to clear isAuthorititative bit on a directory if a file in that
directory is pruned).

> s3guard: implement time-based (TTL) expiry for LocalMetadataStore
> -----------------------------------------------------------------
>
>                 Key: HADOOP-13649
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13649
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>            Priority: Minor
>
> LocalMetadataStore is primarily a reference implementation for testing.  It may be useful
in narrow circumstances where the workload can tolerate short-term lack of inter-node consistency:
 Being in-memory, one JVM/node's LocalMetadataStore will not see another node's changes to
the underlying filesystem.
> To put a bound on the time during which this inconsistency may occur, we should implement
time-based (a.k.a. Time To Live / TTL)  expiration for LocalMetadataStore



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message