oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Barber <tom.bar...@meteorite.bi>
Subject Re: GSoC 2017 - Rework OODT Configuration to make use of Zookeeper
Date Thu, 16 Mar 2017 11:25:50 GMT
Hi Imesha,

Basically, just allowing the dynamic registration of file managers so that
new ones can come and go.

Tom

On Thu, Mar 16, 2017 at 10:00 AM, Imesha Sudasingha <imesha.13@cse.mrt.ac.lk
> wrote:

> Hi Tom,
>
> Need a small clarification. In the last reply, you mentioned that file
> managers may be designed to be *transient. *What do you mean by that? A
> quick response would really help me as I'm trying to come up with a design.
>
> Thank you!
>
> *Imesha Sudasingha*
> Undergraduate of Department of Computer Science and  Engineering,
> University of Moratuwa,
> Sri Lanka.
>
> <https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94>
> <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> <https://twitter.com/Imesha94>
>
> On 13 March 2017 at 22:03, Imesha Sudasingha <imesha.13@cse.mrt.ac.lk>
> wrote:
>
> > Hi Tom,
> >
> > Thanks for the clarification. It really helped a lot to fill most of the
> > gaps I had unanswered. I have a draft design in my mind and will post for
> > your review with details soon.
> >
> > As for my current understanding, my task is to implement a mechanism to
> > switch/chose configuration mechanism (between local files and zookeer for
> > the moment) and then allow APIs to query those mechanisms (in the
> zookeeper
> > scenario) for future use (For File Managers to know about other instances
> > and make decisions accordingly). That include synchronization, fetching
> > configuration from zookeeper if an instance go down and come again and so
> > on. Am I correct?
> >
> > I will try to come up with the design I mentioned, so that both of us can
> > discuss more about the improvements and requirements soon.
> >
> > Thank you for the support!
> >
> > Kind Regards,
> > *Imesha Sudasingha*
> > Undergraduate of Department of Computer Science and  Engineering,
> > University of Moratuwa,
> > Sri Lanka.
> >
> > <https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94
> >
> > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> > <https://twitter.com/Imesha94>
> >
> > On 13 March 2017 at 20:19, Tom Barber <tom.barber@meteorite.bi> wrote:
> >
> >> Hi Imesha
> >>
> >> The best way to demonstrate this is to show you a reference build for
> >> OODT..
> >>
> >> https://dl.dropboxusercontent.com/u/8503756/oodt-example.tgz
> >>
> >> If you can grab that archive and extract it, you'll see that is how
> >> currently OODT is deployed.
> >>
> >> If you take the file manager as the reference module inside oodt/filemgr
> >> you'll see
> >>
> >> etc/filemgr.properties
> >>
> >> and
> >>
> >> policy/oodt/*
> >>
> >> These are the configuration locations of a single file manager.
> >>
> >> The problem I want to solve is this:
> >>
> >> a) If I run > 1 file manager they are completely separate and they don't
> >> know of each other. So using Zookeeper will allow multiple file managers
> >> to
> >> register against ZK and learn of each other.
> >>
> >> b) Backing them onto ZK will allow file managers to come and go on the
> >> fly,
> >> providing scaling capabilities or maybe fm's that are designed to be
> >> transient.
> >>
> >> Just for clarity, the ZK configuration will be an optional requirement,
> so
> >> we need to extend the existing configuration mechanism, to support both
> >> the
> >> current config and the ZK config instead of  ZK being mandatory.
> >>
> >> Tom
> >>
> >> On Mon, Mar 13, 2017 at 3:20 AM, Imesha Sudasingha <
> >> imesha.13@cse.mrt.ac.lk>
> >> wrote:
> >>
> >> > Hi Tom,
> >> >
> >> > Related to the conversation on JIRA [1] ;
> >> >
> >> > I went through the CAS-File manager module and I now understand how to
> >> > address ".properties" files. Since you mentioned in JIRA [2] that
> there
> >> are
> >> > occasions where we use *spring configuration files *for configuration
> >> > purposes, I want to have a look at that type of configuration too. Can
> >> you
> >> > give me an example module where spring configuration files are used to
> >> load
> >> > configuration?
> >> >
> >> > Also, will the content of a configuration file exceed 1MB (Maximum
> data
> >> > size allowed in an ZNode) in any case?
> >> >
> >> > Thank you!
> >> >
> >> > [1]
> >> > https://issues.apache.org/jira/browse/OODT-945?focusedCommen
> >> tId=15906204&
> >> > page=com.atlassian.jira.plugin.system.issuetabpanels:
> >> > comment-tabpanel#comment-15906204
> >> >
> >> > [2]
> >> > https://issues.apache.org/jira/browse/OODT-945?focusedCommen
> >> tId=15906524&
> >> > page=com.atlassian.jira.plugin.system.issuetabpanels:
> >> > comment-tabpanel#comment-15906524
> >> >
> >> > Kind Regards,
> >> > *Imesha Sudasingha*
> >> > Undergraduate of Department of Computer Science and  Engineering,
> >> > University of Moratuwa,
> >> > Sri Lanka.
> >> >
> >> > <https://lk.linkedin.com/in/imeshasudasingha>  <
> >> https://github.com/IMS94>
> >> > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> >> > <https://twitter.com/Imesha94>
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message