asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: A vague question
Date Fri, 31 Jul 2015 09:17:13 GMT
The cluster.xml/Cluster file is just used to represent all the
parameters of the AsterixDB instance, like IP and so on.  For example:
https://github.com/apache/incubator-asterixdb/blob/master/asterix-installer/src/main/resources/clusters/demo/demo.xml
. The user has to create it. Basically it represents the instance,
i.e. if you modify it, that isn't quite the same AsterixDB instance
anymore. I actually reused that piece in the YARN port with slight
modification.

On Fri, Jul 31, 2015 at 1:44 AM, Chris Hillery <chillery@hillery.land> wrote:
> Thanks for the detailed reply! That's very useful information for me.
>
> Could you tell me a little bit about the cluster.xml file? I believe
> Managix creates that, although I'm not sure... is/was it created elsewhere?
> What does it contain, and is its lifecycle?
>
> Thanks,
> Ceej
> aka Chris Hillery
>
> On Fri, Jul 31, 2015 at 1:29 AM, Raman Grover <ramangrover29@gmail.com>
> wrote:
>
>> The asterix-events package dates back to fall of 2012. It was required to
>> have a mechanism/framework for causing "events" in an asterixDB cluster
>> either on a adhoc basis or following a set pattern. A failure of an NC or
>> joining of an NC, execution of an AQL statement could be considered as
>> examples of an event.  Such a framework would allow testing/monitoring of
>> an AsterixDB instance under a constructed environment with a set of events
>> scheduled upfront.  Consider running a workload and causing NC failures
>> that follow a pattern or are completely random and automating the complete
>> test environment.
>>
>> The asterix-events framework modeled an event as an execution of a shell
>> script;   you could do absolutely anything on a cluster as/when you desire
>> as long as the action can be written as a script.
>> Events could be scheduled to occur at specific locations or at random
>> nodes. Asterix-events  allowed defining a "pattern" as an XML to capture
>> the periodicity or dependency (e.g. one event would fire another) for a
>> given event.
>>
>> In Jan of 2013, when we requirement for Managix came up, it made sense to
>> not reinvent the wheel; I thought of reusing the asterix-events framework
>> and model the actions taken by Managix when setting up an instance as
>> events occurring across a cluster. Thus creating an AsterixDB instance is
>> actually an asterix-events pattern that has a sequence of events firing
>> across a cluster, these include transfer of binaries to nodes following by
>> starting of CC and NCs etc.
>>
>> Managix is only a client of the asterix-events framework and as such is
>> very thin in itself;  much of the mechanism to execute a particular step on
>> a cluster is enabled and handled by the underling generic asterix-events
>> framework. Managix just provides the specific pattern to execute for each
>>  managix command. You may also think of a "pattern" as a "job" and
>> asterix-events as the framework that executes the pattern/job.
>> Asterix-events framework also allows listening for errors or completion of
>> an event on the cluster. Managix uses this functionality to report an error
>> on when it fails to execute a step on a node (e.g. failing to start NC on a
>> cluster).
>>
>> I am hopeful that asterix-events framework would find other uses within
>> AsterixDB and possibly be picked up others in simulating event(s) and
>> validating the behavior of the system. E.g. cause random node failures in
>> an automated scripted way and evaluate the impact on our strategy for data
>> replication within AsterixDB.
>>
>> Regards,
>> Raman
>>
>>
>>
>>
>>
>>
>> On Thu, Jul 30, 2015 at 11:47 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>
>> > I believe the original purpose was profiling and instrumenting
>> > AsterixDB. It is also used to provide managix's functionality.
>> >
>> > I know a thing or two about it via my YARN adventures, so I'd be happy
>> > to share all I know. Raman is probably the authority though, as he is
>> > the author.
>> >
>> > -Ian
>> >
>> > On Thu, Jul 30, 2015 at 11:02 PM, Chris Hillery <chillery@lambda.nu>
>> > wrote:
>> > > What purpose is served by the asterix-events package? It appears to
>> > provide
>> > > allow introspecting an Asterix instance, but I think there's more to it
>> > > than that.
>> > >
>> > > What kind of code would be a client of this package?
>> > >
>> > > I'm particularly interested in the lifecycle of the JAXB class
>> "Cluster"
>> > > and the cluster.xml file which is the marshalled form of that class.
>> > >
>> > > Who would be a good person to have a Skype chat with about this?
>> > >
>> > > Thanks,
>> > > Ceej
>> > > aka Chris Hillery
>> >
>>
>>
>>
>> --
>> Raman
>>

Mime
View raw message