karaf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brad Johnson" <bradj...@redhat.com>
Subject Opinionated...
Date Fri, 13 Jan 2017 18:05:29 GMT
Folks,

 

I wanted to make sure that my promoting CDI, Camel Java DSL, & static
profiles didn't obscure the point I was trying to make.  Whatever mechanics
we choose I'd really like us to be unified behind a common paradigm so that
our documentation, exemplars, archetypes, blogs, libraries, and so on are
all organized the same and use the same mechanics and layouts for projects. 

 

We should promote an idiomatic way to develop software using Karaf Boot.
That's one problem I hear from a lot of clients.  There are such
cross-currents of information about how to develop OSGi-based software that
it gets confusing.  Best or preferred practices are lost in the noise.  I
won't get into all that since I'm sure most of you have dealt this problem.
Not to pick on it but a good example is that the Camel in Action book
recommends using Pojos instead of using Processors/Exchanges.  It is on
somewhere near the back of the book in a few pages. I don't know how many
examples on the web site actually use the Processor/Exchange but it is a
lot. Then there are examples with Spring, Blueprint, Java DSL, Scala, etc.
There are annotations that only work in one environment but not in all of
them.

 

By selecting an idiomatic and "opinionated" way of creating Karaf Boot
microcontainers we could make sure that sort of confusion isn't continued
forward.  It would require a lot less documentation to cover the same ground
and make editing and updating easier.  It would make creating sample and
example projects a lot easier. It would simplify what Karaf Boot appliances
have to support and make sure there aren't concerns that work in one
environment and not in another or that might work differently in a different
environment.

 

I'm personally interested in Karaf Appliances with standard Maven
structures, standard  bundle structures, and reference implementations that
have a good chunk of the basic functionality. I'd say we take a page from
the "convention over configuration" book or, at least, a "conventional
configuration" and likely a bit of both. Because the appliances are focused
on microservices we should get out ahead of the Gartner hype cycle.  Right
now we are at the Peak of Inflated Expectations and in a couple of years
we'll be at the Trough of Disillusionment.  That disillusionment will come
for a number of reasons. Flying Spaghetti Monster topology will be one of
them but, more importantly for a Karaf Appliance, is the consistent problem
of "network fallacies".  Every Karaf Kontainer should have standard OSGi
service interfaces and basic implementations that address each of the
fallacies that apply to a uService.  The Kontainers should insist on it and
not make it optional. If the user doesn't want that functionality they would
then need to disable via configuration.  But the Kontainer will get stuck in
a grace period and then fail if an expected, standard service isn't
available. All of the standard OSGi service APIs would have basic
implementations to start but as more specific Kontainers.  But, because they
are standard services new ones can be developed by the community or by the
end developer.

 

As developers, we've all had to implement functionality and then come back
and deal with error handling, security, etc. I say we simply cut those
services in to the Kontainer right from the get-go.  The Kontainer doesn't
run if it doesn't find the service.  That isn't to say these become a
fundamental part of Karaf but a fundamental part of the Kontainer service
that runs in Karaf.

 

The standard bundles would only implement basic functionality and not do
anything sophisticated.  New bundles and libraries for more sophisticated
implementations could be added later. All of the bundles would likely have
disable flags if the developer found the particular concern irrelevant.  For
example, security might not be relevant. The following aren't meant to be
comprehensive. Just addressing key concerns. Other standards like
LoggingService might be included by default as well. 

 

The intent here isn't to define the exact mechanics but the standard OSGi
service interface that would be _required_ in any implementation of a
Kontainer, even if the implemented bundle is simply a passthrough or can be
disabled, it forces the developer to explicitly deal with the problems or
choose to ignore them altogether.  

 

Because these service interfaces and the bundles that implement them are
standard, the set can be specified by the dependencies specified in the
Maven build, features and/or profiles.

 

1.	The network is reliable.

A standard "Error Handler" OSGi service.  The default bundle would simply
capture errors/exceptions and log them.  Perhaps it would specify retries.
Drop in solutions might include errors going to dead letter queues and so
on. The OSGi service interface is required for Kontainer bootstrap so use
the default or use a standard one or create one of your own.  If they want
to change configuration of this bundle or put in a new one, they know
exactly what it is, where it exists, how it is specified to the build, and
what configuration file is associated with it. No rummaging around through
code.  When the inevitable error, exceptions and problems arise, the
developer isn't left wondering where and how they should add the
functionality to handle it.

 

A standard "Circuit Breaker" service API and basic implemented bundle should
be provided.  Perhaps the standard bundle would simply count errors over a
time frame and shut down if that limit is hit and allow those values to be
configured. Default would be a rather unsophisticated implementation but
provide the convention and automated wiring of a circuit breaker OSGi
service.  Other implementations might fire off emails to Sys Admins or be
combinations. And if it is really undesirable, set a disable flag.

 

2.	Latency is zero.

A standard OSGi Throttling service interface and bundle implementation would
be included.  If you want different behavior, change it.  If you want to
disable it, set the flag. However, there are bigger issues here that I'll
address a bit more down below.

 

3.	Bandwidth is infinite.

Throttling OSGi service again. Ditto to comment 2.

 

4.	The network is secure.

Standard OSGi service to plug in in various authentication/authorization
mechanisms.  By default it might be pass through but also have a different
implementation that uses a simple username/password. Obviously LDAP, JAAS,
and other bundles could be created and dropped into place. 

 

5.	Topology doesn't change.

Back to the Circuit Breaker, logging and perhaps notification mechanism.
Also the transport issue below where I'll mention some configuration.

 

6.	There is one administrator.

//No particular plugin for this but standardized configuration and expected
bundles help and this also relates to the transport discussion.

 

7.	Transport cost is zero.

//Probably not a concern here directly but will be a big issue of uServices.

 

8.	The network is homogeneous.

//I think this issue can be dealt with in our context with many of the
standard libraries but can be abstracted a bit more.

 

Obviously a big issue we'll see, and I've seen in the past, is chained
request/response calls. Service 1 making a REST call to service 2 making a
REST call to service 3.etc.  And all of a sudden the latency is a killer.

 

ServiceMix/Karaf/Camel can already abstract away some of that via property
substitution. I'd suggest we take that one step further and put _all_
transport/protocol information in configuration and create a standardized
URI. As a developer or a senior developer over a group of developers, I
don't want them to be concerned with the fiddly bits of the transport in the
code and routes and I certainly don't want to recompile just to make such
changes.

 

Akka, for  example, uses local URIs like akka://.  But a similar Karaf/Camel
URI could be used and mapped via the configuration files.  So the developer
would always use karaf:// in their routes and configuration mapping would
use the URI specified.  karaf://myserviceName.  In the configuration file
might be mapped a transport.configuration.cfg file.

 

I believe that is important for a lot of reasons.  A mid-level or
junior-level developer shouldn't be involved in configuration like:

"ftp://foo@myserver? <ftp://foo@myserver/?> password=secret&amp;

           recursive=true&amp;

           ftpClient.dataTimeout=30000&amp;

           ftpClientConfig.serverLanguageCode=fr"

 

So the cfg file might look like this:

clientService="ftp://foo@myserver?password=secret&

           recursive=true&

           ftpClient.dataTimeout=30000&

           ftpClientConfig.serverLanguageCode=fr"

(At least properties get rid of the gawdaful escaped ampersands).

 

The code would then say "karaf://clientService"

 

One can do much of that via configuration right now but I think it is
critical to move it completely to configuration so that admins know exactly
what to change and where to find it when topologies change. It also means
that when the backlash from microservice calling microservice calling
microservice being slow happens, that simple mapping would permit things
like going to JMS asynchronous request/response (or other fast, async
mechanisms) that don't swamp the virtual machine's or Karaf instance
resources. It would also allow for easy stubbing or mock testing of the
Kontainer as it will be deployed without using PAX exam or other mechanism.

 

Creating standard OSGi service APIs in an anticipation of these problems
would permit for an evolutionary approach to these problems in the future
and specific solutions when a standard Kontainer is developed. Even standard
error handler service implementations can be created.

 

Once such a basic, standard Kontainer exists, then uKontainers that
implement basic functionality commonly used could be created.  There are JPA
examples already.  But the average developer is going to be given a task to
receive some canonical data model via a REST service and poke it into a
database.  That database model probably won't look like what they are
receiving.  So a uKontainer that has a REST front end they can modify, a
Dozer object mapping file in the middle with a transform, and a call to the
database will be used repeatedly.

 

It may be that Oracle, MySQL, BerklyDB, and so on each endup with different
error handler plugin implementations which are used with the same REST,
mapping, JPA container. Just change the Maven dependency or profile.

 

There are a large number of examples like that.  In the case of that
uKontainer there would like be a JPAErrorService for catching common errors
and another for Dozer errors and for unmarshaling errors.  As a developer
looking to solve very specific problems, I just download the uContainer and
do the Dozer mapping, change some configuration and then test it.

 

That also means, that much like Camel EIPs, open source developers can focus
on hardening these containers, fixing bugs, putting in performance
enhancements and the like.  If a new error is coming from JPA that a user
finds and isn't being handled in a coherent fashion, then a new block or
delegate code is added and released.  Just as we'd do with a Camel endpoint
or component.

 

Having standard error handlers built into uKontainers would also help make
coherent messages from the large and unwieldy stack traces full of
reflection that we commonly see.  The error handler OSGi plugin for a given
problem would be highly focused on identifying and reporting problems with a
specific technology or set of technologies.

 

 

 

https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing


Mime
View raw message