oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imesha Sudasingha <imesha...@cse.mrt.ac.lk>
Subject Re: GSoC 2017 - Rework OODT Configuration to make use of Zookeeper
Date Thu, 16 Mar 2017 10:00:20 GMT
Hi Tom,

Need a small clarification. In the last reply, you mentioned that file
managers may be designed to be *transient. *What do you mean by that? A
quick response would really help me as I'm trying to come up with a design.

Thank you!

*Imesha Sudasingha*
Undergraduate of Department of Computer Science and  Engineering,
University of Moratuwa,
Sri Lanka.

<https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94>
<http://stackoverflow.com/users/4012073/imesha-sudasingha>
<https://twitter.com/Imesha94>

On 13 March 2017 at 22:03, Imesha Sudasingha <imesha.13@cse.mrt.ac.lk>
wrote:

> Hi Tom,
>
> Thanks for the clarification. It really helped a lot to fill most of the
> gaps I had unanswered. I have a draft design in my mind and will post for
> your review with details soon.
>
> As for my current understanding, my task is to implement a mechanism to
> switch/chose configuration mechanism (between local files and zookeer for
> the moment) and then allow APIs to query those mechanisms (in the zookeeper
> scenario) for future use (For File Managers to know about other instances
> and make decisions accordingly). That include synchronization, fetching
> configuration from zookeeper if an instance go down and come again and so
> on. Am I correct?
>
> I will try to come up with the design I mentioned, so that both of us can
> discuss more about the improvements and requirements soon.
>
> Thank you for the support!
>
> Kind Regards,
> *Imesha Sudasingha*
> Undergraduate of Department of Computer Science and  Engineering,
> University of Moratuwa,
> Sri Lanka.
>
> <https://lk.linkedin.com/in/imeshasudasingha>  <https://github.com/IMS94>
> <http://stackoverflow.com/users/4012073/imesha-sudasingha>
> <https://twitter.com/Imesha94>
>
> On 13 March 2017 at 20:19, Tom Barber <tom.barber@meteorite.bi> wrote:
>
>> Hi Imesha
>>
>> The best way to demonstrate this is to show you a reference build for
>> OODT..
>>
>> https://dl.dropboxusercontent.com/u/8503756/oodt-example.tgz
>>
>> If you can grab that archive and extract it, you'll see that is how
>> currently OODT is deployed.
>>
>> If you take the file manager as the reference module inside oodt/filemgr
>> you'll see
>>
>> etc/filemgr.properties
>>
>> and
>>
>> policy/oodt/*
>>
>> These are the configuration locations of a single file manager.
>>
>> The problem I want to solve is this:
>>
>> a) If I run > 1 file manager they are completely separate and they don't
>> know of each other. So using Zookeeper will allow multiple file managers
>> to
>> register against ZK and learn of each other.
>>
>> b) Backing them onto ZK will allow file managers to come and go on the
>> fly,
>> providing scaling capabilities or maybe fm's that are designed to be
>> transient.
>>
>> Just for clarity, the ZK configuration will be an optional requirement, so
>> we need to extend the existing configuration mechanism, to support both
>> the
>> current config and the ZK config instead of  ZK being mandatory.
>>
>> Tom
>>
>> On Mon, Mar 13, 2017 at 3:20 AM, Imesha Sudasingha <
>> imesha.13@cse.mrt.ac.lk>
>> wrote:
>>
>> > Hi Tom,
>> >
>> > Related to the conversation on JIRA [1] ;
>> >
>> > I went through the CAS-File manager module and I now understand how to
>> > address ".properties" files. Since you mentioned in JIRA [2] that there
>> are
>> > occasions where we use *spring configuration files *for configuration
>> > purposes, I want to have a look at that type of configuration too. Can
>> you
>> > give me an example module where spring configuration files are used to
>> load
>> > configuration?
>> >
>> > Also, will the content of a configuration file exceed 1MB (Maximum data
>> > size allowed in an ZNode) in any case?
>> >
>> > Thank you!
>> >
>> > [1]
>> > https://issues.apache.org/jira/browse/OODT-945?focusedCommen
>> tId=15906204&
>> > page=com.atlassian.jira.plugin.system.issuetabpanels:
>> > comment-tabpanel#comment-15906204
>> >
>> > [2]
>> > https://issues.apache.org/jira/browse/OODT-945?focusedCommen
>> tId=15906524&
>> > page=com.atlassian.jira.plugin.system.issuetabpanels:
>> > comment-tabpanel#comment-15906524
>> >
>> > Kind Regards,
>> > *Imesha Sudasingha*
>> > Undergraduate of Department of Computer Science and  Engineering,
>> > University of Moratuwa,
>> > Sri Lanka.
>> >
>> > <https://lk.linkedin.com/in/imeshasudasingha>  <
>> https://github.com/IMS94>
>> > <http://stackoverflow.com/users/4012073/imesha-sudasingha>
>> > <https://twitter.com/Imesha94>
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message