Mailing-List: contact dev-help@oodt.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@oodt.apache.org
MIME-Version: 1.0
In-Reply-To: <CABML50cV+mdFz7m5iV1w5fJaVp9cg6=QH3zu77j-GDQx=vgAgw@mail.gmail.com>
References: <CALjF9ONES_nYtGqEXUdm59mbqaGWz+25zLC0tp6-uvTBdwmV1g@mail.gmail.com>
 <CABML50cgnkS0Kw2X1vjGiH4E6A9302YYBh69rPeF1JOPLXLn9A@mail.gmail.com>
 <CALjF9OPP0TUkkkHew+gO6Sgxybv=pny0T5hpG-yD-af8kf3LDQ@mail.gmail.com>
 <CALjF9OOFKu8Fr5J7SScpLbZ0UFNm52TC2OJtBf__Q8EjZ9RHbg@mail.gmail.com>
 <CABML50f2X-hpD_g-v+MW4-vKJVXXRpCLGzDe9Lnk8ZM0zExwWg@mail.gmail.com>
 <CALjF9OPg99tSf4EEgXLE-0oRRPo2q6-eowkkPLM4LeKY-vdyrA@mail.gmail.com>
 <CALjF9OMU3viJZH3vy2t_QRJeHzK+DJ8t1ptm9LRrB5=azUe3AQ@mail.gmail.com>
 <CABML50fw8ohA9i-E+Qv_XggpKD2y_E7qGKUSPxW+Ogjm-2vhRg@mail.gmail.com>
 <CALjF9OPinAre6dey-do0RjaikEAameMQaNbUzG7masG6F_4BKA@mail.gmail.com>
 <CALjF9OO=f9Y590sB5BSXfhWK4ng_Tv6rH+fFamv-+U+1SL+yPw@mail.gmail.com>
 <CALjF9OPXwRRJ6MEhc9_-qZxK_UDPgjGs_SksO3895oOW8EuTxQ@mail.gmail.com>
 <CALjF9OPDTLSVj8v8VzhEd8UJ=4GEvq1YBTc8WXha4hsXNdCgWQ@mail.gmail.com>
 <CABML50cpHOUySvEu85uoiuQjoPNcR3FBqDhj3upu=-ZFa=c3FA@mail.gmail.com>
 <CALjF9OPh3uOC98taKGnzQjWL2uKG05TphMT26tZ8Y5fa2UU+Xw@mail.gmail.com> <CABML50cV+mdFz7m5iV1w5fJaVp9cg6=QH3zu77j-GDQx=vgAgw@mail.gmail.com>
From: Imesha Sudasingha <imesha.13@cse.mrt.ac.lk>
Date: Tue, 28 Mar 2017 08:46:45 +0530
Message-ID: <CALjF9OM0kgg8RorKOT2RuiXo2Cf5y4Ay6PVxp2BEHoJfmHw6rw@mail.gmail.com>
Subject: Re: GSoC 2017 - Rework OODT Configuration to make use of Zookeeper
To: dev@oodt.apache.org
Content-Type: multipart/alternative; boundary=001a1143d3962686d4054bc1e65f
archived-at: Tue, 28 Mar 2017 03:17:33 -0000

--001a1143d3962686d4054bc1e65f
Content-Type: text/plain; charset=UTF-8

Thanks a lot Tom!

Kind Regards,
*Imesha Sudasingha*
Undergraduate of Department of Computer Science and  Engineering,
University of Moratuwa,
Sri Lanka.

<https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94>
<http://stackoverflow.com/users/4012073/imesha-sudasingha>
<https://twitter.com/Imesha94>

On 28 March 2017 at 08:40, Tom Barber <tom.barber@meteorite.bi> wrote:

> Thanks Imesha
>
> I shall check it tomorrow and get back to you ASAP.
>
> Tom
>
> On Tue, Mar 28, 2017 at 2:33 AM, Imesha Sudasingha <
> imesha.13@cse.mrt.ac.lk>
> wrote:
>
> > Hi Tom,
> >
> > As you suggested, I have created a copy of my proposal at [1] and shared
> > with comment permissions. It will be a great help if you can review it
> > sooner and provide a kind feedback.
> >
> > Thank you!
> >
> > [1]
> > https://docs.google.com/document/d/1bnFEVViCwMACHJRB1OiYZ-
> > GAywCNJRxrUJPpHttpgpE/edit?usp=sharing
> >
> > Kind Regards,
> > *Imesha Sudasingha*
> > Undergraduate of Department of Computer Science and  Engineering,
> > University of Moratuwa,
> > Sri Lanka.
> >
> > <https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94
> >
> > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > <https://twitter.com/Imesha94>
> >
> > On 28 March 2017 at 01:13, Tom Barber <tom.barber@meteorite.bi> wrote:
> >
> > > Hi Imesha
> > >
> > > I'm trying to figure out if/how I can get access to it on the
> dashboard.
> > In
> > > the mean time if you want to pastebin it, chuck it in a word doc, or
> > > something completely different I'd be happy to take a look.
> > >
> > > Tom
> > >
> > > On Sun, Mar 26, 2017 at 3:09 PM, Imesha Sudasingha <
> > > imesha.13@cse.mrt.ac.lk>
> > > wrote:
> > >
> > > > Hi Tom,
> > > >
> > > > Did you get a chance to look at my proposal?
> > > >
> > > > Kind Regards,
> > > > *Imesha Sudasingha*
> > > > Undergraduate of Department of Computer Science and  Engineering,
> > > > University of Moratuwa,
> > > > Sri Lanka.
> > > >
> > > > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > https://github.com/IMS94
> > > >
> > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > <https://twitter.com/Imesha94>
> > > >
> > > > On 23 March 2017 at 10:40, Imesha Sudasingha <
> imesha.13@cse.mrt.ac.lk>
> > > > wrote:
> > > >
> > > > > Hi Tom,
> > > > >
> > > > > I have written a draft proposal and shared it with the Apache
> > Software
> > > > > Foundation through the GSoC dashboard. If you get a chance, can you
> > > > please
> > > > > review it and let me know what are the improvements required to be
> > > made.
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > > > Kind Regards,
> > > > > *Imesha Sudasingha*
> > > > > Undergraduate of Department of Computer Science and  Engineering,
> > > > > University of Moratuwa,
> > > > > Sri Lanka.
> > > > >
> > > > > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > https://github.com/IMS94
> > > > >
> > > > > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > > <https://twitter.com/Imesha94>
> > > > >
> > > > > On 22 March 2017 at 14:36, Imesha Sudasingha <
> > imesha.13@cse.mrt.ac.lk>
> > > > > wrote:
> > > > >
> > > > >> Hi Tom,
> > > > >>
> > > > >> I think I had missed your point previously. I was referring to
> just
> > > > >> storing the configuration of different components in different
> > ZNodes
> > > > and
> > > > >> querying them later. Then I understood that we also need to have
> > > > >> inheritance at zookeeper level in order to make it easy to
> configure
> > > > OODT
> > > > >> clusters as you have mentioned in the issue [1]. To configure
> > clusters
> > > > of
> > > > >> OODT components with minimum configuration, I will have to review
> my
> > > > design
> > > > >> again and make a lot of improvements. For example, I have to think
> > > > about a
> > > > >> way to store configuration in zookeeper with minimum duplications
> > and
> > > > give
> > > > >> components enough flexibility to initialize with no or minimum
> > manual
> > > > >> configuration.
> > > > >>
> > > > >> Since I have to submit the GSoC proposal before 3rd of April, I
> > think
> > > > >> that I won't have enough time to review the design and make the
> > > > necessary
> > > > >> improvements before that. Will that negatively affect the
> > possibility
> > > > of me
> > > > >> being able to work on this project for GSoC 2017? Please let me
> know
> > > you
> > > > >> opinion.
> > > > >>
> > > > >> Thank you!
> > > > >>
> > > > >> [1] https://issues.apache.org/jira/browse/OODT-945
> > > > >>
> > > > >> Kind Regards,
> > > > >> *Imesha Sudasingha*
> > > > >> Undergraduate of Department of Computer Science and  Engineering,
> > > > >> University of Moratuwa,
> > > > >> Sri Lanka.
> > > > >>
> > > > >> <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > https://github.com/IMS94>
> > > > >>   <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > >> <https://twitter.com/Imesha94>
> > > > >>
> > > > >> On 21 March 2017 at 10:52, Imesha Sudasingha <
> > imesha.13@cse.mrt.ac.lk
> > > >
> > > > >> wrote:
> > > > >>
> > > > >>> Hi Tom,
> > > > >>>
> > > > >>> Sorry for the late reply. Zookeeper is just providing an ZNode
> > > > >>> structure, which can be considered as a tree where a node is
> > accessed
> > > > as
> > > > >>> directories. For example, in [1] the file manager properties of
> the
> > > > >>> instance with name "*node1*" is stored at the ZPath (Path to that
> > > > >>> ZNode) "*/oodt/node1/file-mgr/filemgr.properties*".
> > > > >>>
> > > > >>> Therefore, my idea is that we can make such directories (ZPaths)
> > per
> > > > >>> each instance in our cluster (ex: */oodt/node1 for instance
> > "node1"*,
> > > > */oodt/node2
> > > > >>> for instance "node2"* and so on). Then inside each such path
> > > (assigned
> > > > >>> per node), we will have separate ZNodes for each component. For
> > > > example, in
> > > > >>> */oodt/node1* we will have an ZNode /oodt/node1/file-mgr which
> > stores
> > > > >>> the configurations of file manager of node1. Similarly, an ZNode
> > > > >>> /oodt/node1/res-mgr will store the configuration of the resource
> > > > manager of
> > > > >>> node1.
> > > > >>>
> > > > >>> Hope you got the idea. Since zookeeper is not some kind of a
> graph
> > > > >>> database, I couldn't think of a method to keep inheritance among
> > > > >>> configuration at Zookeeper's side. However, the design I suggest
> > will
> > > > store
> > > > >>> the configuration independently and will allow the
> users/developers
> > > to
> > > > >>> query those configuration. Therefore in my opinion, we should let
> > the
> > > > >>> developer to compare the configuration and make decisions. This
> > way,
> > > > we can
> > > > >>> allow this distributed configuration manager to manage
> > configuration
> > > > of any
> > > > >>> current component or any upcoming component making it more
> > > extend-able.
> > > > >>>
> > > > >>> Can you let me know what do you think of it?
> > > > >>>
> > > > >>> Thank you!
> > > > >>>
> > > > >>> [1] https://drive.google.com/open?id=
> 0B415Bnipaf1lNThKYXZMN3pFemc
> > > > >>>
> > > > >>> Kind Regards,
> > > > >>> *Imesha Sudasingha*
> > > > >>> Undergraduate of Department of Computer Science and  Engineering,
> > > > >>> University of Moratuwa,
> > > > >>> Sri Lanka.
> > > > >>>
> > > > >>> <https://lk.linkedin.com/in/imeshasudasingha>
> > > > >>> <https://github.com/IMS94>
> > > > >>> <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > >>> <https://twitter.com/Imesha94>
> > > > >>>
> > > > >>> On 20 March 2017 at 05:02, Tom Barber <tom.barber@meteorite.bi>
> > > wrote:
> > > > >>>
> > > > >>>> Hi Imesha
> > > > >>>>
> > > > >>>> Sorry for the delay, I've been travelling and only just got
> myself
> > > > >>>> through
> > > > >>>> my backlog.
> > > > >>>>
> > > > >>>> The overall design looks good. I was also thinking it would make
> > > sense
> > > > >>>> if
> > > > >>>> possible to have some form of inheritance in the config. So, for
> > > > >>>> example,
> > > > >>>> if you have 5 file managers, and they all share the same config
> > > > except a
> > > > >>>> base url or whatever, it would make sense to have a single set
> of
> > > > >>>> configs
> > > > >>>> and then individual overrides for configs that change. (My
> > knowledge
> > > > of
> > > > >>>> Zookeeper isn't great, I'm assuming something along these lines
> is
> > > > >>>> possible).
> > > > >>>>
> > > > >>>> It would also make sense, I think to register this stuff with
> the
> > > > >>>> resource
> > > > >>>> manager, but I'm not sure how that integration would work quite
> > yet,
> > > > so
> > > > >>>> we'll need to figure that one out.
> > > > >>>>
> > > > >>>> Let me know if you have any questions about the above.
> > > > >>>>
> > > > >>>> Tom
> > > > >>>>
> > > > >>>>
> > > > >>>> On Sat, Mar 18, 2017 at 12:39 PM, Imesha Sudasingha <
> > > > >>>> imesha.13@cse.mrt.ac.lk
> > > > >>>> > wrote:
> > > > >>>>
> > > > >>>> > Hi Tom,
> > > > >>>> >
> > > > >>>> > What do you think of the design? Do you agree on implementing
> > this
> > > > >>>> > configuration management section in a separate module?
> > > > >>>> >
> > > > >>>> > I was thinking on starting on the implementation as a proof of
> > > > >>>> concept. Can
> > > > >>>> > you give me your kind feedback before I start on that?
> > > > >>>> >
> > > > >>>> > Thank you!
> > > > >>>> >
> > > > >>>> > Kind Regards,
> > > > >>>> > *Imesha Sudasingha*
> > > > >>>> > Undergraduate of Department of Computer Science and
> > Engineering,
> > > > >>>> > University of Moratuwa,
> > > > >>>> > Sri Lanka.
> > > > >>>> >
> > > > >>>> > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > >>>> https://github.com/IMS94>
> > > > >>>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > >>>> > <https://twitter.com/Imesha94>
> > > > >>>> >
> > > > >>>> > On 16 March 2017 at 17:51, Imesha Sudasingha <
> > > > imesha.13@cse.mrt.ac.lk
> > > > >>>> >
> > > > >>>> > wrote:
> > > > >>>> >
> > > > >>>> > > Hi Tom,
> > > > >>>> > >
> > > > >>>> > > Thanks for the quick reply. I came up with a small design
> [1].
> > > > >>>> > >
> > > > >>>> > > What I suggest is a separate module for configuration
> > > management,
> > > > >>>> which
> > > > >>>> > > can be extended in future and used by many components as
> well.
> > > > >>>> > >
> > > > >>>> > > As shown in the design [1], File Manager (or any component
> > that
> > > is
> > > > >>>> > willing
> > > > >>>> > > to have this extended functionality of distributed
> > configuration
> > > > >>>> > > management) will have an instance of *ConfigurationManager*
> > > which
> > > > >>>> is an
> > > > >>>> > > interface defined to load configuration (specifically,
> > > .properties
> > > > >>>> files)
> > > > >>>> > > and get configurations of each running instance of the same
> > > > >>>> component (If
> > > > >>>> > > distributed configuration management is enabled, this will
> > > return
> > > > >>>> the
> > > > >>>> > > configuration of all the other instances).
> > > > >>>> > >
> > > > >>>> > > There will be two implementing classes of that interface,
> > > > >>>> > > *StandaloneConfigurationManager* and
> > > > *DistributedConfigurationManag
> > > > >>>> er.*
> > > > >>>> > As
> > > > >>>> > > the name implies, standalone configuration manager is to be
> > used
> > > > if
> > > > >>>> the
> > > > >>>> > > users do not intend to used distributed configuration
> > > management.
> > > > >>>> Also
> > > > >>>> > this
> > > > >>>> > > will be same as the current mechanism of configuration
> > > management.
> > > > >>>> Then
> > > > >>>> > the
> > > > >>>> > > distributed configuration manager will be the implementation
> > > using
> > > > >>>> > > Zookeeper.
> > > > >>>> > >
> > > > >>>> > > At zookeeper level, we will need to have a proper naming
> > > mechanism
> > > > >>>> and a
> > > > >>>> > > structure to store configuration. For example,*
> > > > >>>> > > /${nodeName}/${componentName}/{$configFileName}* will be
> the
> > > > place
> > > > >>>> where
> > > > >>>> > > the configuration file contents will be stored.
> > *getFile(String
> > > > >>>> fileName)
> > > > >>>> > > and getFiles(String directiryName)* of *NodeConfiguration*
> > class
> > > > >>>> will
> > > > >>>> > > return the content of the configuration files and
> > configuration
> > > > >>>> > directories
> > > > >>>> > > respectively. I have demonstrated the ZNode structre I'm
> > > expecting
> > > > >>>> to
> > > > >>>> > > design in [2]
> > > > >>>> > >
> > > > >>>> > > Please note that this is just a very high level overview of
> my
> > > > >>>> design and
> > > > >>>> > > the real difficulty of implementation comes when testing
> this
> > > and
> > > > >>>> > > addressing the edge cases of zookeeper. Also, I have only
> > added
> > > > few
> > > > >>>> > methods
> > > > >>>> > > that came to my mind to the *ConfigurationManager*
> interface.
> > If
> > > > >>>> you find
> > > > >>>> > > any fault in this design or have a better design in mind
> > please
> > > > let
> > > > >>>> me
> > > > >>>> > know
> > > > >>>> > > so that I can optimize this design and do my best to come up
> > > with
> > > > >>>> the
> > > > >>>> > best
> > > > >>>> > > possible solution.
> > > > >>>> > >
> > > > >>>> > > Thank you!
> > > > >>>> > >
> > > > >>>> > > [1] https://drive.google.com/open?id=
> > > 0B415Bnipaf1ldnV1dWt4VVVLam8
> > > > >>>> > > [2] https://drive.google.com/open?id=
> > > 0B415Bnipaf1lNThKYXZMN3pFemc
> > > > >>>> > >
> > > > >>>> > > Kind Regards,
> > > > >>>> > > *Imesha Sudasingha*
> > > > >>>> > > Undergraduate of Department of Computer Science and
> > > Engineering,
> > > > >>>> > > University of Moratuwa,
> > > > >>>> > > Sri Lanka.
> > > > >>>> > >
> > > > >>>> > > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > >>>> https://github.com/IMS94
> > > > >>>> > >
> > > > >>>> > > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > > > >>>> > > <https://twitter.com/Imesha94>
> > > > >>>> > >
> > > > >>>> > > On 16 March 2017 at 16:55, Tom Barber <
> > tom.barber@meteorite.bi>
> > > > >>>> wrote:
> > > > >>>> > >
> > > > >>>> > >> Hi Imesha,
> > > > >>>> > >>
> > > > >>>> > >> Basically, just allowing the dynamic registration of file
> > > > managers
> > > > >>>> so
> > > > >>>> > that
> > > > >>>> > >> new ones can come and go.
> > > > >>>> > >>
> > > > >>>> > >> Tom
> > > > >>>> > >>
> > > > >>>> > >> On Thu, Mar 16, 2017 at 10:00 AM, Imesha Sudasingha <
> > > > >>>> > >> imesha.13@cse.mrt.ac.lk
> > > > >>>> > >> > wrote:
> > > > >>>> > >>
> > > > >>>> > >> > Hi Tom,
> > > > >>>> > >> >
> > > > >>>> > >> > Need a small clarification. In the last reply, you
> > mentioned
> > > > >>>> that file
> > > > >>>> > >> > managers may be designed to be *transient. *What do you
> > mean
> > > by
> > > > >>>> that?
> > > > >>>> > A
> > > > >>>> > >> > quick response would really help me as I'm trying to come
> > up
> > > > >>>> with a
> > > > >>>> > >> design.
> > > > >>>> > >> >
> > > > >>>> > >> > Thank you!
> > > > >>>> > >> >
> > > > >>>> > >> > *Imesha Sudasingha*
> > > > >>>> > >> > Undergraduate of Department of Computer Science and
> > > > Engineering,
> > > > >>>> > >> > University of Moratuwa,
> > > > >>>> > >> > Sri Lanka.
> > > > >>>> > >> >
> > > > >>>> > >> > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > >>>> > >> https://github.com/IMS94>
> > > > >>>> > >> > <http://stackoverflow.com/users/4012073/imesha-
> sudasingha>
> > > > >>>> > >> > <https://twitter.com/Imesha94>
> > > > >>>> > >> >
> > > > >>>> > >> > On 13 March 2017 at 22:03, Imesha Sudasingha <
> > > > >>>> imesha.13@cse.mrt.ac.lk
> > > > >>>> > >
> > > > >>>> > >> > wrote:
> > > > >>>> > >> >
> > > > >>>> > >> > > Hi Tom,
> > > > >>>> > >> > >
> > > > >>>> > >> > > Thanks for the clarification. It really helped a lot to
> > > fill
> > > > >>>> most of
> > > > >>>> > >> the
> > > > >>>> > >> > > gaps I had unanswered. I have a draft design in my mind
> > and
> > > > >>>> will
> > > > >>>> > post
> > > > >>>> > >> for
> > > > >>>> > >> > > your review with details soon.
> > > > >>>> > >> > >
> > > > >>>> > >> > > As for my current understanding, my task is to
> implement
> > a
> > > > >>>> mechanism
> > > > >>>> > >> to
> > > > >>>> > >> > > switch/chose configuration mechanism (between local
> files
> > > and
> > > > >>>> > zookeer
> > > > >>>> > >> for
> > > > >>>> > >> > > the moment) and then allow APIs to query those
> mechanisms
> > > (in
> > > > >>>> the
> > > > >>>> > >> > zookeeper
> > > > >>>> > >> > > scenario) for future use (For File Managers to know
> about
> > > > other
> > > > >>>> > >> instances
> > > > >>>> > >> > > and make decisions accordingly). That include
> > > > synchronization,
> > > > >>>> > >> fetching
> > > > >>>> > >> > > configuration from zookeeper if an instance go down and
> > > come
> > > > >>>> again
> > > > >>>> > >> and so
> > > > >>>> > >> > > on. Am I correct?
> > > > >>>> > >> > >
> > > > >>>> > >> > > I will try to come up with the design I mentioned, so
> > that
> > > > >>>> both of
> > > > >>>> > us
> > > > >>>> > >> can
> > > > >>>> > >> > > discuss more about the improvements and requirements
> > soon.
> > > > >>>> > >> > >
> > > > >>>> > >> > > Thank you for the support!
> > > > >>>> > >> > >
> > > > >>>> > >> > > Kind Regards,
> > > > >>>> > >> > > *Imesha Sudasingha*
> > > > >>>> > >> > > Undergraduate of Department of Computer Science and
> > > > >>>> Engineering,
> > > > >>>> > >> > > University of Moratuwa,
> > > > >>>> > >> > > Sri Lanka.
> > > > >>>> > >> > >
> > > > >>>> > >> > > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > >>>> > >> https://github.com/IMS94
> > > > >>>> > >> > >
> > > > >>>> > >> > > <http://stackoverflow.com/users/4012073/imesha-
> > sudasingha>
> > > > >>>> > >> > > <https://twitter.com/Imesha94>
> > > > >>>> > >> > >
> > > > >>>> > >> > > On 13 March 2017 at 20:19, Tom Barber <
> > > > tom.barber@meteorite.bi
> > > > >>>> >
> > > > >>>> > >> wrote:
> > > > >>>> > >> > >
> > > > >>>> > >> > >> Hi Imesha
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> The best way to demonstrate this is to show you a
> > > reference
> > > > >>>> build
> > > > >>>> > for
> > > > >>>> > >> > >> OODT..
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> https://dl.dropboxusercontent.
> > > > com/u/8503756/oodt-example.tgz
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> If you can grab that archive and extract it, you'll
> see
> > > that
> > > > >>>> is how
> > > > >>>> > >> > >> currently OODT is deployed.
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> If you take the file manager as the reference module
> > > inside
> > > > >>>> > >> oodt/filemgr
> > > > >>>> > >> > >> you'll see
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> etc/filemgr.properties
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> and
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> policy/oodt/*
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> These are the configuration locations of a single file
> > > > >>>> manager.
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> The problem I want to solve is this:
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> a) If I run > 1 file manager they are completely
> > separate
> > > > and
> > > > >>>> they
> > > > >>>> > >> don't
> > > > >>>> > >> > >> know of each other. So using Zookeeper will allow
> > multiple
> > > > >>>> file
> > > > >>>> > >> managers
> > > > >>>> > >> > >> to
> > > > >>>> > >> > >> register against ZK and learn of each other.
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> b) Backing them onto ZK will allow file managers to
> come
> > > and
> > > > >>>> go on
> > > > >>>> > >> the
> > > > >>>> > >> > >> fly,
> > > > >>>> > >> > >> providing scaling capabilities or maybe fm's that are
> > > > >>>> designed to
> > > > >>>> > be
> > > > >>>> > >> > >> transient.
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> Just for clarity, the ZK configuration will be an
> > optional
> > > > >>>> > >> requirement,
> > > > >>>> > >> > so
> > > > >>>> > >> > >> we need to extend the existing configuration
> mechanism,
> > to
> > > > >>>> support
> > > > >>>> > >> both
> > > > >>>> > >> > >> the
> > > > >>>> > >> > >> current config and the ZK config instead of  ZK being
> > > > >>>> mandatory.
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> Tom
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> On Mon, Mar 13, 2017 at 3:20 AM, Imesha Sudasingha <
> > > > >>>> > >> > >> imesha.13@cse.mrt.ac.lk>
> > > > >>>> > >> > >> wrote:
> > > > >>>> > >> > >>
> > > > >>>> > >> > >> > Hi Tom,
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > Related to the conversation on JIRA [1] ;
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > I went through the CAS-File manager module and I now
> > > > >>>> understand
> > > > >>>> > >> how to
> > > > >>>> > >> > >> > address ".properties" files. Since you mentioned in
> > JIRA
> > > > >>>> [2] that
> > > > >>>> > >> > there
> > > > >>>> > >> > >> are
> > > > >>>> > >> > >> > occasions where we use *spring configuration files
> > *for
> > > > >>>> > >> configuration
> > > > >>>> > >> > >> > purposes, I want to have a look at that type of
> > > > >>>> configuration
> > > > >>>> > too.
> > > > >>>> > >> Can
> > > > >>>> > >> > >> you
> > > > >>>> > >> > >> > give me an example module where spring configuration
> > > files
> > > > >>>> are
> > > > >>>> > >> used to
> > > > >>>> > >> > >> load
> > > > >>>> > >> > >> > configuration?
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > Also, will the content of a configuration file
> exceed
> > > 1MB
> > > > >>>> > (Maximum
> > > > >>>> > >> > data
> > > > >>>> > >> > >> > size allowed in an ZNode) in any case?
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > Thank you!
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > [1]
> > > > >>>> > >> > >> > https://issues.apache.org/jira
> > > > >>>> /browse/OODT-945?focusedCommen
> > > > >>>> > >> > >> tId=15906204&
> > > > >>>> > >> > >> > page=com.atlassian.jira.
> plugin.system.issuetabpanels:
> > > > >>>> > >> > >> > comment-tabpanel#comment-15906204
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > [2]
> > > > >>>> > >> > >> > https://issues.apache.org/jira
> > > > >>>> /browse/OODT-945?focusedCommen
> > > > >>>> > >> > >> tId=15906524&
> > > > >>>> > >> > >> > page=com.atlassian.jira.
> plugin.system.issuetabpanels:
> > > > >>>> > >> > >> > comment-tabpanel#comment-15906524
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > Kind Regards,
> > > > >>>> > >> > >> > *Imesha Sudasingha*
> > > > >>>> > >> > >> > Undergraduate of Department of Computer Science and
> > > > >>>> Engineering,
> > > > >>>> > >> > >> > University of Moratuwa,
> > > > >>>> > >> > >> > Sri Lanka.
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >> > <https://lk.linkedin.com/in/imeshasudasingha>  <
> > > > >>>> > >> > >> https://github.com/IMS94>
> > > > >>>> > >> > >> > <http://stackoverflow.com/users/4012073/imesha-
> > > sudasingha
> > > > >
> > > > >>>> > >> > >> > <https://twitter.com/Imesha94>
> > > > >>>> > >> > >> >
> > > > >>>> > >> > >>
> > > > >>>> > >> > >
> > > > >>>> > >> > >
> > > > >>>> > >> >
> > > > >>>> > >>
> > > > >>>> > >
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> >
>

--001a1143d3962686d4054bc1e65f--