distributedlog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <guosi...@gmail.com>
Subject Re: [DISCUSS] DP-8: Symlinked Log
Date Wed, 19 Apr 2017 17:32:17 GMT
Agreed that a general reconfiguration approach will be ideal for solving
the problems

However currently DL is using a metadata store (e.g. ZooKeeper) for storing
and managing the metadata part. So the capabilities of metadata
reconfiguration is kind of limited by the features offered by the metadata
store. For example, zookeeper doesn't support rename operations. A symlink
approach here is try to address the metadata reconfiguration problem when
the underneath metadata store lacks such ability. Beside that, moving a
flat namespace to a hierarchical namespace in general is a long-duration
operation. So an approach should exist to allow both co-exist for a while
and allow rolling back and forward.

- Sijie

On Wed, Apr 19, 2017 at 8:14 AM, Dan Benediktson <dbenediktson@twitter.com>

> It seems to me that the fact that the namespace format needs to be changed
> is the specific, immediate problem, but the bigger problem is that there
> isn't a mechanism to perform an atomic reconfiguration step for log
> metadata. Namespace format change is one example of a reconfiguration (if
> we start viewing namespace format as a part of log metadata, which just
> happens to currently be unspecified since there is no format other than the
> flat one), but there are already other metadata reconfiguration changes
> that are desired: https://issues.apache.org/jira/browse/DL-74 and
> https://issues.apache.org/jira/browse/DL-77. I think we should try to
> solve the general reconfiguration problem correctly first, and then that
> should enable us to solve both this problem and those two more naturally. I
> expect that a correct reconfiguration solution would obviate the need for
> this sub-task: https://issues.apache.org/jira/browse/DL-201.
> On Tue, Apr 18, 2017 at 1:06 PM, Leigh Stewart <lstewart@twitter.com>
> wrote:
>> + Dan
>> On Tue, Apr 18, 2017 at 1:04 PM, Sijie Guo <guosijie@gmail.com> wrote:
>>> Hi all,
>>> I created a proposal for supporting symlinks in Dlog, in order to
>>> migrate a namespace from flat namespace to a hierarchical namespace.
>>> https://cwiki.apache.org/confluence/display/DL/DP-8+-+Symlinked+Log
>>> Please take a look and let me know your thoughts.
>>> - Sijie

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message