incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Urs Lerch <m...@ulerch.net>
Subject Re: [VOTE] ALOIS to enter the incubator
Date Thu, 16 Sep 2010 23:37:25 GMT
Hi Craig

Thanks for you input and your vote.

In my view, the chat is not a must, but a proposition. I understand your
concern and I certainly will keep it in mind. But since ALOIS should
become a community project, I still think the followers of it should
decide which communication channell they prefer. (By the way, I myself
sure am no fan of chats.)

I hope you can live with this.

Best
Urs


Am Donnerstag, den 16.09.2010, 10:49 -0700 schrieb Craig L Russell:
> Hi Urs,
> 
> My only concern is the request to have a chat channel. There's wide  
> use of chat channels in Apache (the periodic board and members'  
> meetings make use of them, and infrastructure uses channels to  
> advantage).
> 
> But for an incubating project, I'd strongly discourage use of chat as  
> a communication channel.
> 
> +1
> 
> Craig
> 
> On Aug 26, 2010, at 9:09 AM, Urs Lerch wrote:
> 
> > Hi,
> >
> > I would like to call a vote for accepting "ALOIS" for incubation in
> > the Apache Incubator. The full proposal is available below and on the
> > proposal wiki page (http://wiki.apache.org/incubator/ 
> > AloisProposal).  We
> > ask the Incubator PMC to sponsor it, with Scott Deboy volunteering as
> > Champion and Mentor. Additional mentors are warmly welcome.
> >
> > Please cast your vote:
> >
> > [ ] +1, bring ALOIS into Incubator
> > [ ] +0, I don't care either way,
> > [ ] -1, do not bring ALOIS into Incubator, because...
> >
> > This vote will be open for 72 hours and only votes from the Incubator
> > PMC are binding.
> >
> > Thanks,
> > Urs
> >
> >
> > --------------------------------------------
> >
> >
> > = Preface =
> >
> > ALOIS is a log collection and correlation software with reporting and
> > alarming functionalities. It has been implemented by the Swiss company
> > IMSEC for a customer about five years ago. GPL-licenced, implemented  
> > in
> > Ruby and completely based on other OSS-licensed components, it was
> > designed for the open source community right from the start. Now that
> > the software has shown its functioning over several years in  
> > production
> > with the one customer and one IMSEC-internal installation, it seems to
> > be the right time to open it to a wider community.
> >
> >
> > = Abstract =
> >
> > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and
> > is meant to be a fully implemented open source SIEM (security
> > information and event management) system.
> >
> >
> > = Proposal =
> >
> > While almost all other SIEM software, be it closed or open source,
> > concentrate on the technological part of security monitoring, ALOIS is
> > aimed to monitor the security of the content. It intends to be
> > pro-active in the detection of potential loss, theft, mistaken
> > modification or unauthorized access. ALOIS works on log messages and
> > thus contains all the basic functionality of a conventional SIEM, as
> > centralized collecting, normalizing, aggregation, analyzing and
> > correlating of all log messages, as well as reporting all security
> > related events. Therefore it can be used as any other SIEM.
> >
> > ALOIS consists of five modules interacting to ensure a scaleable
> > functionality of a SIEM:
> >
> >  * Insink is the message sink, which is the receiving entry point for
> > all the different log messages into ALOIS. It is partly based on the
> > syslog-ng software. Insink listens for messages (UDP), waits for
> > messages (TCP), receives message collections (files, emails) and
> > pre-filters them to prevent from message flow overload.
> >
> >  * Pumpy is the incoming FIFO buffer, implemented as a relational
> > database tables. which contain the incoming original messages (in raw
> > format). In a complex system setup, there may be several insink
> > instances, e.g. for a group of hosts, for specific types of  
> > messages, or
> > for high-avaliablity.
> >
> >  * Prisma contains logic to split up the text of log messages into
> > separate fields, based on regular expressions. Actually, "prisma" is a
> > set of "prismi", each one prisma for one type of log message (apache,
> > cisco etc. Several prismi can be applied to the same message. This
> > allows for stacked messages, i.e. forwarded log messages contained in
> > compressed files contained in e-mail messages. The data retrieved form
> > the log messages is stored in a database called Dobby. Due to prisma
> > being written in Ruby, prismi can be applied interactively (when  
> > having
> > system access).
> >
> >  * Dobby is the central log database. It should be separated from the
> > Pumpy database for availability and performance reasons. The current
> > implementation is based on MySQL.
> >
> >  * The Analyzer contains the two sub-systems Lizard and Reptor. Lizard
> > is the analysis engine and user interface of ALOIS, implemented in  
> > Ruby
> > on Rails using AJAX. It allows for interactive browsing through the
> > collected data, exclusion/inclusion/selection of data, data sorting,
> > data filtering, creation of views, ad-hoc textual and graphical
> > reporting. Reptor allows for automatic activation of views and
> > comparison of these views' results to a predefined result (pattern
> > matching). In case of mismatch, Reptor sends the result to predefined
> > e-mail addresses.
> >
> > Its modular design guarantees ALOIS to scale from little to large
> > organizations. Since there exists a Debian package, it's easy to  
> > build a
> > test system or even a productive system for small environments.
> >
> > Although the software has been in productive use for a few years,  
> > there
> > is still a lot of desired functionality missing. The plugability of  
> > new
> > connected systems is given, but needs some revision. It is a given  
> > goal
> > of the project to allow modules in other programming language.
> > Furthermore, it has been discussed if parts of the existing
> > implementation may be replaced with other proven open source software,
> > e.g. the correlation engine or the web frontend. The other way  
> > round, it
> > has been discussed that the filter creation engine would make a good
> > tool for any kind of structured data, and thus could be separated from
> > ALOIS and standardized as a stand-alone tool.
> >
> >
> > = Background =
> >
> > It's not simple to know what happens in a bigger network. There's a
> > multitude of applications, services and appliances working together.
> > Many of them provide some kind of events or state information. The
> > network administrator needs to get hands on all of them. But they come
> > in many different flavors and multiple canals. Therefore, it's hard to
> > get the big picture. Furthermore, we have learned that it's impossible
> > to protect a system against all malicious attacks and to keep all the
> > possible faulty handling away. A monitoring of the systems to  
> > guarantee
> > a pro-active handling is therefore needed..
> >
> > Therefore, more and more organizations collect and analyze all  
> > logfiles
> > in a centralized system, called a SIEM (security information and event
> > management). The technology provides two major functions for security
> > events from networks, systems and applications: log management and
> > compliance reporting (SIM – security information management) and
> > real-time monitoring and incident management (SEM – security event
> > management).
> >
> >
> > = Rationale =
> >
> > Why another security information and event management system? It's  
> > true,
> > there's already plenty of them. While the proprietary software is way
> > too expensive for smaller to mid-sized companies, we find that the  
> > open
> > source solutions are either too simple or not completely open. For
> > example, behind each of the well known systems “OSSIM” and “Prelude”,
> > there is a company that either closes central functionality for its  
> > own
> > business or has dual licensing and therefore asks the full copyright  
> > for
> > all contributed code.
> >
> > ALOIS is aimed to be totally free and open for all contributions. The
> > openness provided for other programming languages is certainly proof  
> > of
> > this. The plug-ability - yet to be further developed - is meant to
> > guarantee that individual needs can be realized without stressing the
> > whole system too much. In our opinion, the Linux kernel is a good
> > example that this can work very well.
> >
> > Since we are in accordance with „the Apache way“, we would be very
> > pleased if ALOIS could become part of the Apache community. In  
> > Addition,
> > the Apache Logging Services would be a perfect home for the software.
> > Furthermore, it's not the intention to compete with the already  
> > existing
> > log viewer and analyzing tool „Chainsaw“. Since Chainsaw is a  
> > relatively
> > easy tool, it meets a rather different need. Nevertheless, if the two
> > projects use synergies, both can profit.
> >
> >
> > = Initial Goals =
> >
> > When this project started ins 2005, there was no proven SIEM open  
> > source
> > software and the commercial tools were way too expensive for the  
> > needed
> > environment. Therefore, we decided together with a customer of ours to
> > implement an open source SIEM tool from scratch. Now the software has
> > run in a production environment for several years and has proven its
> > functionality and reliabilty.
> >
> >
> > = Current Status =
> >
> > == Meritocracy ==
> >
> > As already mentioned, ALOIS is already in production use in two
> > organizations. All the code has been written by two persons of the  
> > same
> > company in a paid employment relationship. It is obvious that this is
> > way different from the open source approach within Apache. But
> > nevertheless, the two developers have always worked as a team and the
> > decisions were made in consensus whenever possible. But it is no  
> > secret,
> > that these developers have to learn to behave in an open community.
> > Understanding this potential problem, they already got support by a
> > freelance consulter, who has the corresponding experience and  
> > knowledge.
> >
> > == Community ==
> >
> > Until today there is no real community, because the project hasn't  
> > been
> > published officially, although it had been completely published on the
> > web site for a couple of months (until a server relaunch). Convinced  
> > by
> > the concept and design of the software, we are open and hope to reach
> > many contributors and users. We think that it is realistic, because  
> > the
> > SIEM issue has yet not been resolved in the OSS space.
> >
> > == Core Developers ==
> >
> > ALOIS was developed by Simon Hürliman and Flavio Pellanda, both  
> > employed
> > by the company IMSEC. Concerning Design and Architecture, Marcus
> > Holthaus, owner of IMSEC, gave his input as security specialist. Since
> > the beginning of this year, Urs Lerch, a doctorate on the subject of
> > commercial open source software development, supports the team with  
> > his
> > knowledge. Simon Hürlimann has left the company three years ago, but  
> > is
> > still active in the OSS environment (although not for ALOIS). Current
> > employee Daniel Lutz (a Debian Developer) has also contributed to the
> > project.
> >
> > == Alignment ==
> >
> > Besides that we strongly believe in the „Apache way“, we think that
> > although that Apache hosts the Logging Services and different security
> > projects, there is a gap when it comes to a superordinate security  
> > view.
> > We therefore think it a good idea to add our SIEM project to the  
> > Apache
> > repository. On the other side, Apache would become an even more  
> > complete
> > software repository.
> >
> >
> > = Known Risks =
> >
> > == Orphaned products ==
> >
> > Since the software is only maintained by employers of one company,  
> > there
> > is a severe risk of being orphaned. But, on the one hand, the company
> > has a sustained interest in keeping the project alive, because there  
> > are
> > plans to offer services on top of ALOIS, and IMSEC uses the software  
> > for
> > SIEM on their own systems. For this reason there exists a budget for  
> > the
> > development and support of ALOIS. On the other hand, we believe that
> > ALOIS is of great interest for other people and companies tied to IT
> > security. Therefore, our step to the Apache incubator is also a step  
> > to
> > a bigger community.
> >
> > == Inexperience with Open Source ==
> >
> > While ALOIS has always been licenced under the GPL, access to the  
> > source
> > code, bug tracker and version control system has been restricted to
> > internal users for most of the time. But the company has a strong
> > believe in the open source movement and therefore engages its  
> > employees
> > to take part in the community. Furthermore, it is also a strategic
> > decision to build services on top of linux.
> >
> > We understand that the Apache Incubator is a great opportunity for  
> > us to
> > get assistance, when it comes to specific questions on the open source
> > development. Even more, the company has created a part time position  
> > for
> > the open source community work.
> >
> > == Homogenous Developers ==
> >
> > Although ALOIS has been developed by employees of only one company,
> > there is a thorough openness. The company is designed to stay small  
> > and
> > therefore works with several independent partners. Furthermore, its
> > employees work in geographically different parts of the country.
> > Therefore, it is no new experience for the developers to work in a
> > distributed environment and argue rather than to command. Already  
> > today
> > the employees are enforced to document all face-to-face  
> > communication in
> > the internal wiki. Sketches are photographed and stored in the  
> > project's
> > digital folder.
> >
> > == Reliance on Salaried Developers ==
> >
> > Until today all the development of ALOIS has been made in a paid
> > emplyoment. Therefore we know that this brings a significant danger.
> > Since it is our stated aim to encourage participation and recruit
> > commiters, we hope to eliminate this risk as soon as possible.
> > Furthermore, the employees of IMSEC are all open source enthusiasts  
> > and
> > are in one way or another active in the community. Although we have no
> > certainty, there is good indication that the current commiters would
> > continue their work on ALOIS, even if they wouldn't be paid for it.
> >
> > == Relationships with Other Apache Products ==
> >
> > The Apache Logging Service would be a perfect home for ALOIS as a
> > centralized logging collection and analyzing tool. Furthermore, we  
> > think
> > that we could share part of the code with the Chainsaw subproject,  
> > since
> > both need similar functionality in the web frontend. Since it is our
> > statet aim to replace our own code with proofen open source libraries,
> > we are open for any collaboration with other projects. For example,  
> > the
> > replacement of the MySQL with a NoSQL database might be useful for
> > performance reasons; therefore HBase is a good candidate.
> >
> > == An Excessive Fascination with the Apache Brand ==
> >
> > The Apache brand is in fact for its own a very good reason to join the
> > Incubator. But much more our desire to become part of the Apache
> > Incubator is our strong believe in open source software in general and
> > in the „Apache way“ in particular. We would love to learn from the
> > experience and knowledge of the foundation's members and participants,
> > which is an important part of the brand as well. The foundation has
> > shown many times, that it has the processes and people to succeed in
> > launching a project. We would be very proud to be part of this success
> > story.
> >
> >
> > = Documentation =
> >
> > The documentation is rather weak and scattered. It has mainly been
> > maintained on a wiki and is open to improvement. Since we are totally
> > aware that this is a killer for a successfull open source project, we
> > have already started an internal project with its own budget to  
> > improve
> > this shortcomming. Once the project has been launched, writing a  
> > blog or
> > open a forum are other possibilities we already thought of.
> >
> > Furthermore, as the employees are used to work in a geographycally
> > distributed environment, a lot of the internal communication happens  
> > in
> > a chat. Thus, opening a new chat channel for the community is  
> > scheduled.
> > (To document the discussions for all those who were off-line, we would
> > send the logs daily to the mailing list.)
> >
> >
> > = Initial Source =
> >
> > Although the initial source comes from a project for a customer. it  
> > has
> > an open source licence since the beginning. Therefore it doesn't have
> > any propriatary code in it. A thorough revision before releasing it  
> > to a
> > public repository is recommend and is also in planning.
> >
> > The initial source will be a snapshot of the version control system,
> > accompanied by a related debian package.
> >
> >
> > = Source and Intellectual Property Submission Plan =
> >
> > ALOIS is currently under a GPL licence. Since there are only two
> > contributors so far, both from the same company, there is no problem  
> > to
> > re-licence the code and contribute it to Apache. The commitment of the
> > company's owner has been granted.
> >
> >
> > = External Dependencies =
> >
> > So far, no external dependencies are known. As mentioned before, a
> > thorough revision of the codebase is in planning. There it can be
> > controlled, that no other licence is affected by the code.
> >
> >
> > = Cryptography =
> >
> > ALOIS does not involve cryptographic code.
> >
> >
> > = Required Resources =
> >
> > == Mailing lists ==
> >
> > The following mailing lists will be required:
> >
> >  * alois-private
> >  * alois-dev
> >  * alois-commits
> >  * alois-users
> >
> > == Subversion Directory ==
> >
> > https://svn.apache.org/repos/asf/incubator/alois
> >
> > == Issue Tracking ==
> >
> > JIRA ALOIS (ALOIS)
> >
> > == Other Resources ==
> >
> > We would like to open a chat channel. If this isn't possible within  
> > the
> > infrastructure of Apache, we would love to do this in our own already
> > existing infrastructure.
> >
> >
> > = Initial Commiters =
> >
> >  * NAME             EMAIL                              AFFILIATION   
> > CLA
> >  * Flavio Pellanda  flavio.pellanda at logintas dot ch IMSEC        no
> >  * Urs Lerch        mail at ulerch dot net             IMSEC        no
> >  * Daniel Lutz      daniel.lutz at logintas dot ch     IMSEC        no
> >  * Marcus Holthaus  marcus.holthaus at imsec dot ch    IMSEC        no
> >
> >
> > = Sponsors =
> >
> > == Champion ==
> >
> >  * Scott Deboy <sdeboy at apache dot org>
> >
> > == Nominated Mentors ==
> >
> >  * Scott Deboy <sdeboy at apache dot org>
> >
> > == Sponsoring Entity ==
> >
> > The Incubator PMC (requested) b
> 
> Craig L Russell
> Architect, Oracle
> http://db.apache.org/jdo
> 408 276-5638 mailto:Craig.Russell@oracle.com
> P.S. A good JDO? O, Gasp!
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
> 


Mime
View raw message