incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <>
Subject [Incubator Wiki] Update of "Plc4XProposal" by ChristoferDutz
Date Sat, 09 Dec 2017 21:51:14 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "Plc4XProposal" page has been changed by ChristoferDutz:

Replaced the copied page content with the real content for the plc4x proposal.

- ## page was copied from AccumuloProposal
- = Accumulo Proposal =
- == Abstract ==
+ = Abstract =
- Accumulo is a distributed key/value store that provides expressive, cell-level access labels.
+ PLC4X is intended to become a set of libraries for communicating with industrial programmable
logic controllers (PLCs) using a variety of protocols but with a shared API.
- == Proposal ==
- Accumulo is a sorted, distributed key/value store based on Google's BigTable design.  It
is built on top of Apache Hadoop, Zookeeper, and Thrift.  It features a few novel improvements
on the BigTable design in the form of cell-level access labels and a server-side programming
mechanism that can modify key/value pairs at various points in the data management process.
+ For the most used of these protocols, PLC4X will provide implementations of drivers. Each
protocol driver is hereby provided by an individual artifact.
+ In a first step, we are concentrating on Java but will be hoping to be able to port implementation
to C and other languages.
- == Background ==
+ = Background =
- Google published the design of BigTable in 2006.  Several other open source projects have
implemented aspects of this design including HBase, CloudStore, and Cassandra.  Accumulo began
its development in 2008.
+ Industrial machines have been controlled by PLCs for more than 30 years. In simpler pieces
of machinery these may operate autonomously, in more complex ones several to hundreds of these
industrial computers communicate with each other.
- == Rationale ==
+ = Rationale =
- There is a need for a flexible, high performance distributed key/value store that provides
expressive, fine-grained access labels.  The communities we expect to be most interested in
such a project are government, health care, and other industries where privacy is a concern.
 We have made much progress in developing this project over the past 3 years and believe both
the project and the interested communities would benefit from this work being openly available
and having open development.
+ Currently software for communicating with PLCs is mainly proprietary software and a whole
ecosystem of closed-source solutions exist. Usually this software is very expensive and licensing
implies a lot of restrictions on its usage. There is a small set of open-source libraries
available, but unfortunately most of these are GPL licensed and hereby disqualify themselves
from commercial use. Most of these are direct ports from C/C++ implementations and therefore
inherit their API. Usually these are implemented blocking socket operations and have great
problems with concurrency. Also, the APIs of these tools differ quite a lot, so porting a
software from communicating with one type of PLC to another sometimes requires re-writing
a great part of the software.
+ There are multiple rationales behind this project:
+ 1. By providing libraries with Apache license, it will be possible to create commercial
applications which access PLCs
+ 2. By providing a universal API for different protocols, it reduces the vendor lock-in when
creating software
+ 3. Most of the proprietary commercial solutions are based on Windows systems, especially
the Siemens solutions, sometimes require maintaining un-patched versions of older Windows
version in order to run, this is a huge security risk (See Stuxnet)
+ 4. Most of the proprietary commercial solutions don't scale. Running them in public/private
clouds and/or in containers is completely out of the question.
+ As a result of above, it would be possible to start writing secure, scalable and reliable
software using the full stack of open-source solutions and hereby open a complete new market
for developers. In order to make this even easier, we are planning on directly providing adapters
and examples for using PLC4X together with other Apache frameworks.
+ = Initial Goals =
+ Develop a universal API for accessing industrial programmable logic controllers in a protocol-agnostic
way. Also implement Java versions of drivers for the most prominent protocols: S7, Modbus
and OPC-UA - but not limited to those. On the side of the adapters, for existing OS solutions,
we are currently working on adapters for: Apache Edgent (incubating), Apache Camel and eventually
even Apache Brooklyn.
+ Also, we will be providing a Scala wrapper to ease integration of PLC4X in Scala based systems
with an API Scala developers should easily adopt.
+ As soon as a C version exists, we are planning on providing an Apache Mynewt adapter as
we see great benefits on this combination.
- == Current Status ==
+ = Current Status =
- === Meritocracy ===
+ == Meritocracy ==
- We intend to strongly encourage the community to help with and contribute to the code. 
We will actively seek potential committers and help them become familiar with the codebase.
+ PLC4X was initially created by Christofer Dutz. He is an Apache Member and also involved
in a variety of Apache and Non-Apache Open-Source projects. Being PMC of the Apache Flex,
Apache Kibble and the Apache Travel Assistance he definitely knows how Apache works and is
a strong advocate of the Apache Way. He is also currently helping the Apache Edgent project
as member of the PPMC. However, the team working on PLC4X also includes a diverse set of additional
Apache Committers and Members as well as people, that are not yet affiliated with any Apache
project. The team working on PLC4X are both working for codecentric as well as other entities,
such as commercial companies as well as universities.
- === Community ===
+ == Community ==
- A strong government community has developed around Accumulo and training classes have been
ongoing for about a year.  Hundreds of developers use Accumulo.
+ Currently the PLC4X community exists out of a set of core developers working on a private
code repository provided by codecentric (the company Christofer Dutz is working for). However,
the set of developers, working on PLC4X is not limited to codecentric employees as we are
also inviting outside contributors.
+ The main reason for using a private repo instead of a public one, is that we were planning
on starting incubation at Apache within a few weeks after initiating the project. Therefore,
we wanted to start writing everything with Apache package names and Apache Maven coordinates.
Even our logo is directly designed including the Apache Feather and "Apache" inside the name.
As only real Apache projects are allowed to use "Apache", we decided to go down the private
repository path.
- === Core Developers ===
+ == Core Developers ==
- The developers are mainly employed by the National Security Agency, but we anticipate interest
developing among other companies.
+ (In alphabetical order)
+ - Benedikt Ritter (codecentric) (Apache Member)
+ - Christofer Dutz (codecentric) (Apache Member)
+ - Dale LaBossiere (IBM) (PPMC Member of the Apache Edgent incubator podling)
+ - Frank Stibane (codecentric)
+ - Greg Trasuk (Web Age Solutions) (Apache Member)
+ - Justin McLean (External) (Apache Member)
+ - Kai Herings (codecentric)
+ - Lukas Friedrichsen (Hochschule Bochum)
+ - Mark Keinhörster (codecentric)
+ - Rolf Felker (codecentric)
+ - Sebastian Rühl (codecentric)
- === Alignment ===
+ == Alignment ==
- Accumulo is built on top of Hadoop, Zookeeper, and Thrift.  It builds with Maven.  Due to
the strong relationship with these Apache projects, the incubator is a good match for Accumulo.
+ The build system is based on Apache Maven. Also, it uses Apache Edgent (incubating) and
Apache Camel libraries for providing adapters to these systems. Being a full-blown Apache
enthusiast, for Christofer there hasn't been any thought in bringing this project into any
other collaboration platform or foundation. This idea is shared by a big majority of the core
+ == Project Name ==
+ As we know that renaming a codebase can be a huge pain, we have invested quite a lot of
time into finding a suitable name. In the end, we came up with a list of about 20 project
names, which we did the fully required background checks for. On the options surviving these
checks, we internally voted on the name. It was a pretty close tie between PLC4J and PLC4X,
but we decided to go for the later as we are planning on providing libraries for multiple
languages beyond Java.
+ PLC4X has no registered trademarks that we could find and the only potential problem would
be a hardware LPT Breakout Board by a Russian Company called Purelogic (
English version (
+ But we don't see any problems to arise from this. If there would be problems, we would probably
fallback to using PLC4J, which had no conflicts at all and would eventually name our sub-products
PLC4S (Scala) and PLC4C (C/C++).
- == Known Risks ==
+ = Known Risks =
- === Orphaned Products ===
+ == Orphaned projects ==
- There is only a small risk of being orphaned.  The community is committed to improving the
codebase of the project due to its fulfilling needs not addressed by any other software.
+ PLC4X is a new Project, which has just been started. The feedback from the industry has
been overwhelming so the risk of the project being orphaned is minimal.
- === Inexperience with Open Source ===
+ == Inexperience with Open Source ==
- The codebase has been treated internally as an open source project since its beginning,
and the initial Apache committers have been involved with the code for multiple years.  While
our experience with public open source is limited, we do not anticipate difficulty in operating
under Apache's development process.
+ The set of core developers of this project consists of a large set of experienced open-source
developers, most of these even being Apache Members.
- === Homogeneous Developers ===
+ == Homogeneous Developers ==
- The committers have multiple employers and it is expected that committers from different
companies will be recruited.
+ Even if a big part of the projects initial committer team is affiliated with codecentric,
we have been successful and continuing to be in actively recruiting people from other commercial
and non-commercial entities.
- === Reliance on Salaried Developers ===
+ == Reliance on Salaried Developers ==
- The initial committers are all paid by their employers to work on Accumulo and we expect
such employment to continue.  Some of the initial committers would continue as volunteers
even if no longer employed to do so.
+ Currently only Christofer Dutz is being paid to work on this project exclusively.
- === Relationships with Other Apache Products ===
+ == Relationships with Other Apache Products ==
- Accumulo uses Hadoop, Zookeeper, Thrift, Maven, log4j, commons-lang, -net, -io, -jci, -collections,
-configuration, -logging, and -codec.
+ PLC4X makes great used of Apache Maven for the build system and Apache Commons for providing
a lot of basic functionality.
- === Relationship to HBase ===
- Accumulo and HBase are both based on the design of Google's BigTable, so there is a danger
that potential users will have difficulty distinguishing the two.  Some of the key areas in
which Accumulo differs from HBase are discussed below.  It may be possible to incorporate
the desired features of Accumulo into HBase.  However, the amount of work required would slow
development of HBase and Accumulo considerably.  We believe this warrants a podling for Accumulo
at the current time.  We expect active cross-pollination will occur between HBase and podling
Accumulo and it is possible that the codebases and projects will ultimately converge.
+ Also does it currently have direct relations to the Apache Camel and Apache Edgent projects
as it provides modules for direct integration of PLC4X into these frameworks.
- ==== Access Labels ====
- Accumulo has an additional portion of its key that sorts after the column qualifier and
before the timestamp.  It is called column visibility and enables expressive cell-level access
control.  Authorizations are passed with each query to control what data is returned to the
user.  The column visibilities are boolean AND and OR combinations of arbitrary strings (such
as "(A&B)|C") and authorizations are sets of strings (such as {C,D}).
+ In the future we are planning on also adding such modules for Apache Brooklyn and as soon
as a C-based version of PLC4X is available Apache Mynewt.
- ==== Iterators ====
- Accumulo has a novel server-side programming mechanism that can modify the data written
to disk or returned to the user.  This mechanism can be configured for any of the scopes where
data is read from or written to disk.  It can be used to perform joins on data within a single
+ Currently there is no competing project at the Apache Software Foundation.
- ==== Flexibility ====
- HBase requires the user to specify the set of column families to be used up front.  Accumulo
places no restrictions on the column families.  Also, each column family in HBase is stored
separately on disk.  Accumulo allows column families to be grouped together on disk, as does
BigTable.  This enables users to configure how their data is stored, potentially providing
improvements in compression and lookup speeds.  It gives Accumulo a row/column hybrid nature,
while HBase is currently column-oriented.
+ == An Excessive Fascination with the Apache Brand ==
- ==== Testing ====
- Accumulo has testing frameworks that have resulted in its achieving a high level of correctness
and performance.  We have observed that under some configurations and conditions Accumulo
will outperform HBase and provide greater data integrity.
+ We are certain that PLC4X itself will profit from being an official Apache project, in terms
of attracting a community and establishing a solid group of developers and users, but also
to ease the interaction with other Apache projects, used in PLC4X or project PLC4X provides
adapters for as all will share the same base.
- ==== Logging ====
- HBase uses a write-ahead log on the Hadoop Distributed File System.  Accumulo has its own
logging service that does not depend on communication with the HDFS NameNode.
+ Our general belief in the Apache way as well as the reasons mentioned above are the main
reasons for us to send this proposal. We think that a good community is needed to build and
maintain good software, such as PLC4X. Also do we need a big community to support a big variety
of systems. However, even if PLC4X would not be accepted, development would definitely continue
elsewhere. Apache is just our primary foundation of choice. As such, there is no need to,
or reason to, "abuse" the Apache Brand.
- ==== Storage ====
- Accumulo has a relative key file format that improves compression.
+ == Possibility of legal actions ==
- ==== Areas in which HBase features improvements over Accumulo ====
- in memory tables, upserts, coprocessors, connections to other projects such as Cascading
and Pig
+ Some of the protocols are not publicly defined. Existing open-source implementations contain
licensing restrictions. To avoid licensing issues we made sure the license of information
used to implement drivers in PLC4X are compatible with the Apache license and added references
to these sources in the project documentation to prove this. This documentation is part of
the codebase. We are therefore not expecting any real problems on this side.
- === Expectations ===
- There is a risk that Accumulo will be criticized for not providing adequate security.  The
access labels in Accumulo do not in themselves provide a complete security solution, but are
a mechanism for labeling each piece of data with the authorizations that are necessary to
see it.
+ == Test setup could require hardware ==
- === Apache Brand ===
- Our interest in releasing this code as an Apache incubator project is due to its strong
relationship with other Apache projects, i.e. Accumulo has dependencies on Hadoop, Zookeeper,
and Thrift and has complementary goals to HBase.
+ A test-suite for PLC4X could involve tests run against virtual or real hardware. It could
be problematic to include these in automated builds run on, but Apache infra
have already signaled that there should be options to work this out.
- == Documentation ==
+ = Documentation =
- There is not currently documentation about Accumulo on the web, but a fair amount of documentation
and training materials exists and will be provided on the Accumulo wiki at  Also,
a paper discussing YCSB results for Accumulo will be presented at the 2011 Symposium on Cloud
+ Currently all documentation and information is stored alongside the code in a private corporate
GIT repository and is available as part of the website which is generated as part of the build.
- == Initial Source ==
+ = Initial Source =
- Accumulo has been in development since spring 2008.  There are hundreds of developers using
it and tens of developers have contributed to it.  The core codebase consists of 200,000 lines
of code (mainly Java) and 100s of pages of documentation.  There are also a few projects built
on top of Accumulo that may be added to its contrib in the future.  These include support
for Hive, Matlab, YCSB, and graph processing.
+ Development of PLC4X started at codecentric in October of 2017. The source currently is
located on a private corporate GIT repository. All the code is available at
(Accounts can be created on a request basis)
- == Source and Intellectual Property Submission Plan ==
+ = Source and Intellectual Property Submission Plan =
- Accumulo core code, examples, documention, and training materials will be submitted by the
National Security Agency.
- We will also be soliciting contributions of further plugins from MIT Lincoln Labs, Carnegie
Mellon University, and others.
+ PLC4X is currently mainly developed by, but not limited to, codecentric employees. However,
all code has been developed completely from scratch it was ensured from the start that each
document is already fully licensed under the Apache 2.0 license. All source will be donated
to Apache without any exceptions.
- Accumulo has been developed by a mix of government employees and private companies under
government contract.  Material developed by government employees is in the public domain and
no U.S. copyright exists in works of the federal government.  For the contractor developed
material in the initial submission, the U.S. Government has sufficient authority per the ICLA
from the copyright owner to contribute the Accumulo code to the incubator.
- There has been some discussion regarding accepting contributions from US Government sources
on We propose that the NSA will sign an ICLA/CCLA
if that document could be slightly modified to explicitly address copyright in works of government
employees. Specifically, we propose that the definition of “You” be modified to include
“the copyright owner, the owner of a Contribution not subject to copyright, or legal entity
authorized by the copyright owner that is making this Agreement.” In addition, section 2,
the copyright license grant be modified after “You hereby grant” that either states “to
the extent authorized by law” or “to the extent copyright exists in the Contribution.”
 These changes will permit US Government employee developed work to be included.
- One proposed solution is to form a Collaborative Research and Development Agreement (CRADA)
between the Apache Software Foundation and the US Government, but this will not solve the
underlying problem that U.S. law does not grant copyright to works of government employees.
 At this time a CRADA is not necessary but should it be determined that a CRADA is necessary,
we would like to work through that process during the incubation phase of Accumulo rather
than before acceptance as this may take time to enter into an agreement.
- == External Dependencies ==
+ = External Dependencies =
- jetty (Apache and EPL), jline (BSD), jfreechart (LGPL), jcommon (LGPL), slf4j (MIT), junit
+ - Apache Commons-IO, Apache License 2.0
+ - Apache Edgent, Apache License 2.0
+ - Netty, Apache License 2.0
+ - Metrics Core, Apache License 2.0
+ - Google Gson, Apache License 2.0
+ This source can be included, according to
+ - Logback, EPL 1.0
+ This source can NOT be included, according to
- == Cryptography ==
+ = Cryptography =
- none
+ Not applicable.
- == Required Resources ==
+ = Required Resources =
-  * Mailing Lists
+ == Mailing Lists ==
-    * accumulo-private
-    * accumulo-dev
-    * accumulo-commits
-    * accumulo-user
-  * Subversion Directory
-    *
+ - plc4x-dev
+ - plc4x-private
+ == Git Repository ==
-  * Issue Tracking
+ == Issue Tracking ==
-    * JIRA Accumulo (ACCUMULO)
-  * Continuous Integration
-    * Jenkins builds on
+ == Confluence Wiki ==
-  * Web
-    *
-    * wiki at or
+ To be able to provide help, documentation, faq etc, a wiki is needed.
- == Initial Committers ==
+ = Initial Committers =
-  * Aaron Cordova (aaron at cordovas dot org)
-  * Adam Fuchs (adam.p.fuchs at ugov dot gov)
-  * Eric Newton (ecn at swcomplete dot com)
-  * Billie Rinaldi (billie.j.rinaldi at ugov dot gov)
-  * Keith Turner (keith.turner at ptech-llc dot com)
-  * John Vines (john.w.vines at ugov dot gov)
-  * Chris Waring (christopher.a.waring at ugov dot gov)
+ (In alphabetical order)
+ - Benedikt Ritter (
+ - Christofer Dutz (
+ - Dale LaBossiere (
+ - Frank Stibane (
+ - Greg Trasuk (
+ - Justin McLean (
+ - Kai Herings (
+ - Lukas Friedrichsen (
+ - Mark Keinhörster (
+ - Rolf Felker (
+ - Sebastian Rühl (
- == Affiliations ==
+ = Affiliations =
-  * Aaron Cordova, The Interllective
-  * Adam Fuchs, National Security Agency
-  * Eric Newton, SW Complete Incorporated
-  * Billie Rinaldi, National Security Agency
-  * Keith Turner, Peterson Technology LLC
-  * John Vines, National Security Agency
-  * Chris Waring, National Security Agency
+ Currently most of the initial committers are employees of codecentric. Currently Christofer
Dutz is the only contributor paid to work on PLC4X full-time. The others participate as part
of their so-called "+1 time" (At codecentric employees get to spend 20% of their time to work
on open-source, education, etc. and they invest a lot of this time on the project).
+ Besides these we also have contributors from companies other than codecentric (Dale, Greg,
Justin) as well as universities (Lukas).
- == Sponsors ==
+ = Sponsors =
-  * Champion: Doug Cutting
+ == Champion ==
+ - Justin McLean
  == Nominated Mentors ==
-  * Benson Margulies
-  * Alan Cabrera
-  * Bernd Fondermann
-  * Owen O'Malley
+ (In alphabetical order)
+ - Greg Trasuk
+ - Justin McLean
+ - Luciano Resende
+ - Stefan Bodewig
  == Sponsoring Entity ==
-  * Apache Incubator
+ PLC4X is a new project and proposed is to release to code under the sponsorship of the Incubator.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message