asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raman Grover <ramangrove...@gmail.com>
Subject Re: A vague question
Date Fri, 31 Jul 2015 08:29:49 GMT
The asterix-events package dates back to fall of 2012. It was required to
have a mechanism/framework for causing "events" in an asterixDB cluster
either on a adhoc basis or following a set pattern. A failure of an NC or
joining of an NC, execution of an AQL statement could be considered as
examples of an event.  Such a framework would allow testing/monitoring of
an AsterixDB instance under a constructed environment with a set of events
scheduled upfront.  Consider running a workload and causing NC failures
that follow a pattern or are completely random and automating the complete
test environment.

The asterix-events framework modeled an event as an execution of a shell
script;   you could do absolutely anything on a cluster as/when you desire
as long as the action can be written as a script.
Events could be scheduled to occur at specific locations or at random
nodes. Asterix-events  allowed defining a "pattern" as an XML to capture
the periodicity or dependency (e.g. one event would fire another) for a
given event.

In Jan of 2013, when we requirement for Managix came up, it made sense to
not reinvent the wheel; I thought of reusing the asterix-events framework
and model the actions taken by Managix when setting up an instance as
events occurring across a cluster. Thus creating an AsterixDB instance is
actually an asterix-events pattern that has a sequence of events firing
across a cluster, these include transfer of binaries to nodes following by
starting of CC and NCs etc.

Managix is only a client of the asterix-events framework and as such is
very thin in itself;  much of the mechanism to execute a particular step on
a cluster is enabled and handled by the underling generic asterix-events
framework. Managix just provides the specific pattern to execute for each
 managix command. You may also think of a "pattern" as a "job" and
asterix-events as the framework that executes the pattern/job.
Asterix-events framework also allows listening for errors or completion of
an event on the cluster. Managix uses this functionality to report an error
on when it fails to execute a step on a node (e.g. failing to start NC on a
cluster).

I am hopeful that asterix-events framework would find other uses within
AsterixDB and possibly be picked up others in simulating event(s) and
validating the behavior of the system. E.g. cause random node failures in
an automated scripted way and evaluate the impact on our strategy for data
replication within AsterixDB.

Regards,
Raman






On Thu, Jul 30, 2015 at 11:47 PM, Ian Maxon <imaxon@uci.edu> wrote:

> I believe the original purpose was profiling and instrumenting
> AsterixDB. It is also used to provide managix's functionality.
>
> I know a thing or two about it via my YARN adventures, so I'd be happy
> to share all I know. Raman is probably the authority though, as he is
> the author.
>
> -Ian
>
> On Thu, Jul 30, 2015 at 11:02 PM, Chris Hillery <chillery@lambda.nu>
> wrote:
> > What purpose is served by the asterix-events package? It appears to
> provide
> > allow introspecting an Asterix instance, but I think there's more to it
> > than that.
> >
> > What kind of code would be a client of this package?
> >
> > I'm particularly interested in the lifecycle of the JAXB class "Cluster"
> > and the cluster.xml file which is the marshalled form of that class.
> >
> > Who would be a good person to have a Skype chat with about this?
> >
> > Thanks,
> > Ceej
> > aka Chris Hillery
>



-- 
Raman

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message