polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niclas Hedhman <nic...@hedhman.org>
Subject Re: Application Code Management tool
Date Sun, 03 May 2015 03:19:45 GMT
Had a quick look, and although there are a few things that I don't like, I
would like to focus on the "Intent" and agree that it would be good if
there is an essence of a generic scaffold here. The code is, after all,
just an implementation detail (those of you who hasn't been around since
the start, the Qi4j Core is on its 4th complete rewrite), and don't feel
offended if we are ending up far from the current position.

So a "Domain Capability" is a end-to-end application stack, complete with
stores/index/config at the bottom and service/http layer at top?? And each
DC is self-deployable?
What is a "Block" more precisely? It seems to be one layer, or possibly
multiple layers, no sure...

My guess is that the REST part here is the central point of abstraction,
and that is pretty cool on one hand, but we would need to make the REST
aspect a generic abstraction.

Thinking out loud (assuming I am getting the main point of Tower);

Tower.forCapabilityClass( RestApplication.class ) // defines layers and
major scaffolding
   .onPort( HTTP_PORT )   // Type-safe from the RestApplication generic
   .addBlock( Timetables.class )  // RestApplication needs a super type
with DSL in it
   .using( new InMemoryEntityStore() )  // Why not ordinary Assemblers?
   .using( new ElasticSearchMemoryIndexing() ).identifiedBy(
"elasticsearch" )
   .start( Application.Mode.test );

public class RestApplication extends CapabilityClass
{
    public RestApplication onPort( int port )
    {
        this.port = port;
    }

    @Layer( "connectivity" )
    @UsesLayers( { "domain", "security" } )
    public class ConnectivityLayer
    {

        @Module( "rest" )
        public void onStart( ModuleAssembly assembly )
        {
        }
    }
}


>From the above, another idea struck; What happens if one can define the
application structure as a single interface;

@Application
public interface MyApplication
{
    @Layer
    @Uses( ServiceLayer.class )
    interface ConnectivityLayer
    {
        @Module
        @Assemblers( { RestliAssembler.class, JettyAssembler.class } )
        interface RestModule {}
    }

    @Layer
    @Uses(DomainLayer.class)
    @Assemblers( DynamicServicesAssembler.class )  // in case the modules
are
                                                   // not known until
runtime
    interface ServiceLayer{}

    @Layer
    interface DomainLayer{}
     :
     :
}

And pass that to the ApplicationAssemblyFactory, or even a wrapper around
the whole thing at a higher level. That could actually be very useful
starting point. Of course the devil is in the details, but by avoid
replicating Assembly constructs (as current Tower does to some extent) we
might not need to sacrifice the power of flexibility, yet get something
easier to comprehend initially.

Cheers

On Sat, May 2, 2015 at 5:43 PM, Michael Pantazoglou <
michael.pantazoglou@gmail.com> wrote:

> Hi Niclas, all,
>
> I set up a temp repo in Github to share the code for this Tower utility.
>
> https://github.com/michaelpantazoglou/smartnerds-tower
>
> Unfortunately the code is not build-able, as it depends on some additional
> artefacts, but hopefully it will give you a better idea of what its
> functionality is intended for.
>
> We are using Tower extensively in our JUnit test cases in order to quickly
> assemble and launch a Qi4j app with all needed composites etc. An example
> is as follows:
>
> @Before
> public void setUp() throws Exception {
>
>    tower = SmartnerdsTower.newInstance(Application.Mode.test);
>
>  tower.addDomainCapabilityBlock(SmartnerdsCatalogue.DomainCapability.Timetables,
>          SmartnerdsTowerConstants.DEFAULT_CONFIGURATION_PROPERTIES,
>          SmartnerdsCatalogue.EntityStore.Memory,
>          SmartnerdsTowerConstants.DEFAULT_CONFIGURATION_PROPERTIES,
>          SmartnerdsCatalogue.EntityIndex.ElasticSearchInMemory,
>          SmartnerdsTowerConstants.DEFAULT_CONFIGURATION_PROPERTIES
>    ).start(HTTP_PORT);
>
>    module =
> tower.getModule(SmartnerdsCatalogue.DomainCapability.Timetables.getId());
>
>    restLiClient = RestLiClient.Factory.createRetLiClient(HTTP_HOST,
> HTTP_PORT);
> }
>
> @After
> public void tearDown() throws Exception {
>
>    restLiClient.shutdown();
>    tower.stop();
> }
>
> Here we start a Tower in test mode, hosting one Domain Capability. A Tower
> can host as many blocks as needed. For each such block, the following
> parameters are expected:
>
> 1) Domain Capability Id: This reference is used to retrieve details about
> the capability from an internal catalogue, where we register all
> capabilities. Those details include the Capability assembler, and the
> resource packages that rest.li will look into to deploy the capability's
> REST API.
>
> 2) Capability Config Properties: This is the name of the (optional)
> .properties file that is used to configure the capability (not all
> capabilities require configuration, but some do)
>
> 3) Entity Store Id: Again, this reference is used to retrieve the assembler
> of the entity store that this capability will use. Note that, we are using
> again our internal catalogue, where we register each ES for which we have
> an assembler.
>
> 4) Entity Store Config Properties: The name of the (optional) .properties
> file that may be required in order to configure the ES (e.g. see PostgreSQL
> ES)
>
> 5) Entity Index Id: Similar semantics like 3), but applies to the EI
>
> 6) Entity Index Config Properties: Similar semantics like 4), but applies
> to the EI
>
> The .start(HTTP_PORT) simply starts the rest.li server that will offer the
> REST API of that capability through the specified port.
>
> * * *
>
> So with this utility we can easily assemble and deploy as many building
> blocks as required by a particular use case, and we can easily configure
> different sets of ES/EI infrastructure to be used by each building block
> independently. We can even use many Tower instances on the same JVM to
> provision parts of our system through different ports, configurations, etc.
>
> From our point of view, it would be great if, out of this custom piece of
> code, we could derive an independent, generic mechanism that would help any
> entry-level Qi4j developer get their code up and running fast - hence make
> them feel the Zest! :-)
>
> Cheers,
> Michael
>
>
> On Tue, Apr 28, 2015 at 2:48 AM, Niclas Hedhman <niclas@hedhman.org>
> wrote:
>
> > Michael,
> > This sounds really cool, and the kind of innovation that I was hoping
> Qi4j
> > would enable when we set out on this long and bumpy journey. There is
> > definitely a lot of generic value here, and I think we just need to get a
> > deeper understanding exactly what it is.
> >
> > So, what are those five parameters in addBlock() ?? Is it an arbitrary
> list
> > of Assemblers, or something very specific in each?
> >
> > I would like to suggest;
> > If you could post the relevant API or sources somewhere, I would like to
> > take a deeper look. For me, I don't need it to be "pretty" and be
> > buildable... (You guys should have seen what Rickard sent me pre-Qi4j
> that
> > became the starting point for this project's initial steps.)
> >
> > Unless you have some IP Rights issues, just create a "temporary" repo on
> > GitHub, and I (hopefully others) will take a closer look.
> >
> >
> > Looking forward to something new and exciting.
> >
> >
> > // Niclas
> >
> > On Tue, Apr 28, 2015 at 3:55 AM, Michael Pantazoglou <
> > michael.pantazoglou@gmail.com> wrote:
> >
> > > Hi guys,
> > >
> > > Apologies for picking up the discussion that Jiri opened a few days
> ago,
> > > perhaps I am already off-topic.
> > >
> > > Tower is an abstraction/utility that we devised to address the
> underlying
> > > complexity of assembly/layers/visibility. At least for me, proper
> > assembly
> > > of infrastructure and visibility of entities across layers was one of
> the
> > > 'hard' things to digest when I started using Qi4j, and at times it gave
> > > frustrating errors. Tower alleviates from a lot of details and provides
> > us
> > > with a kind of DSL of the form:
> > >
> > > Tower.newInstance(Application.Mode)
> > >         .addBlock(A, ES_a, ES_a_config, EI_b, EI_b_config)
> > >         .addBlock(B, ES_x, ES_x_config, EI_y, EI_y_config)
> > >         ...
> > >         .start(httpPort);
> > >
> > > The above would assemble an application with two *blocks* (each block
> is
> > > roughly a domain model + business logic/usecases), with each using the
> > > specified ES and EI infrastructure, and would expose the respective
> REST
> > > API on the specified host and port. All the details are hidden by
> > > identifiable assemblers which reflectively assemble all needed
> components
> > > (entities, transients, values, services etc.) and can be easily reused.
> > So
> > > the whole idea is like building a Tower (aka a Qi4j application) with
> > lego
> > > pieces (the infrastructure blocks and the capability blocks).
> > >
> > > Of course the DSL is tailored to our particular code structure and
> needs,
> > > and the Tower has a few tight dependencies with both ours and some
> > > third-party code, e.g. with rest.li. But if people here are
> interested,
> > we
> > > could explore ways to enhance it, make it more abstract and generic,
> > > perhaps even use it somehow along with the archetype builders that are
> > > being discussed through this thread.
> > >
> > > In general, we are open to ideas and willing to contribute.
> > >
> > > Cheers,
> > > Michael
> > >
> > >
> > > On Fri, Apr 24, 2015 at 2:39 PM, Jiri Jetmar <juergen.jetmar@gmail.com
> >
> > > wrote:
> > >
> > > > We are surely willing to donate it - its a way to say thank you to
> the
> > > > Qi4j/Apache Zest
> > > > Community. :-)
> > > >
> > > > Ok, the Tower code was mostly written by Michael Pantazoglou so I
> need
> > to
> > > > talk with him
> > > > what would be the best approach to donate it. I guess we have to
> solve
> > > some
> > > > dependencies and some package naming stuff, etc.
> > > >
> > > > @Michael - could you pls take the lead here and discuss with me and
> the
> > > > Apache Zest
> > > > community the best best approach to donate the code, clean the
> > > > dependencies, etc ?
> > > >
> > > > Thank you.
> > > >
> > > > Cheers,
> > > > Jiri
> > > >
> > > >
> > > > 2015-04-24 13:15 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:
> > > >
> > > > > Maybe. I am not sure, until I see some more details. Are you
> willing
> > to
> > > > > donate it?
> > > > >
> > > > > I want to see that one can create Qi4j applications, where I list
> > what
> > > > > layers, modules and (ready-made) assemblers I want. And literally
> > have
> > > a
> > > > > fully-structured, Restful HelloWorld running in 1 minute, complete
> > with
> > > > > build system and a re-usable 'model'.
> > > > >
> > > > > Tower might be useful, I would like to know more...
> > > > >
> > > > >
> > > > > Welcome back, Jiri.
> > > > >
> > > > > On Fri, Apr 24, 2015 at 6:34 PM, Jiri Jetmar <
> > juergen.jetmar@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Guys,
> > > > > >
> > > > > > we did in the past something similar to structure & deploy
our
> Qi4j
> > > > Apps.
> > > > > > We are using the Term "Tower"
> > > > > > that contains all relevant components to deliver a service to
the
> > > > client.
> > > > > >
> > > > > > This components are :
> > > > > >
> > > > > > - (State Model) - Domain Model describes the possible states
of a
> > > > > solution.
> > > > > > It also exposes a DM API that is used by above layers.
> > > > > > -  (Behavioral Model) - Context, Roles
> > > > > > - (Interaction Model) Service REST API. We are using here
> > > > > http://rest.li/
> > > > > > - Configuration & Bootstrapping (e.g. initial Data)
> > > > > > - (Infrastructure) EntityStore + Index, Logging (Fluentd setup),
> > > > > Monitoring
> > > > > > - (Deployment) Docker based approach to "bake" docker images
> > > > > > - Gradle based approach to glue, structure & automate the
above
> > > > approach
> > > > > > and steps
> > > > > >
> > > > > > The whole approach is like to peel an onion :
> > > > > >
> > > > > > Hardware > OS > Docker > App > Tower(s) > Qi4j
Modules and layers
> > > > > >
> > > > > > It took as a while to bring all the stuff working together but
> now
> > it
> > > > is
> > > > > > pretty robust and it is
> > > > > > fun to use it..
> > > > > >
> > > > > > I;m not sure Niclas if this  goes to the direction you are
> thinking
> > > on
> > > > -
> > > > > I
> > > > > > mean in
> > > > > > terms of simplifying and decreasing the entry level for future
> Qi4j
> > > > > > hackers. We needed
> > > > > > an approach to push Qi4j-based Apps to production in a robust,
> > > > automated
> > > > > > and
> > > > > > easy way.
> > > > > >
> > > > > > Thank you.
> > > > > >
> > > > > > Cheers,
> > > > > > Jiri
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2015-04-24 5:16 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:
> > > > > >
> > > > > > > Gang,
> > > > > > > After the presentation in Romania, one of the feedbacks
> received
> > > was
> > > > > that
> > > > > > > it is too hard to get going with Qi4j. Not only does it
require
> > > > quite a
> > > > > > > steep learning curve to grasp Qi4j itself, but it is tedious
to
> > set
> > > > up
> > > > > a
> > > > > > > working build for a new project.
> > > > > > >
> > > > > > > So, I want to create something similar to Maven Archetypes,
but
> > > with
> > > > > much
> > > > > > > better understanding of Qi4j structures.
> > > > > > >
> > > > > > > I have created a branch for this; Gradle_archetype_toolchain
> > > > > > > Name was set before I realize what I want to do, but Gradle
> will
> > be
> > > > the
> > > > > > > first supported build system, but I think at least Maven
should
> > > also
> > > > be
> > > > > > > supported, and possibly be able to create Eclipse Workspaces
> and
> > > > > IntelliJ
> > > > > > > projects as well.
> > > > > > >
> > > > > > > Problem domain;
> > > > > > >   + Support Pre-packaged application structures, i.e.
> templating
> > > > > > >   + Support creation/removal of all Qi4j primary types,
> > > Application,
> > > > > > Layer,
> > > > > > > Module, Composites
> > > > > > >   + Support weaving in custom code, so generation can occur
> more
> > > than
> > > > > > once.
> > > > > > >   + Support generation to many different build tools.
> > > > > > >
> > > > > > > Solution domain;
> > > > > > >   * Strong domain model, which is kept in an entity store
and
> > > > modified
> > > > > > > interactively or via scripting
> > > > > > >   * Set of commands for manipulating the model
> > > > > > >   * The entire entity store can be used as a "template"
for new
> > > > > projects
> > > > > > >   * Generators will use the model and generate the structures
> > > > > > >   * Commands are also used to start generation
> > > > > > >
> > > > > > > Example Use-case 1
> > > > > > > Developer Alex want to use Qi4j for a RESTful server
> application.
> > > He
> > > > > > isues
> > > > > > > the 'create-project' command and selects the 'rest-server'
> > > > application
> > > > > > > type, and the tool creates a operational skeleton application
> > that
> > > > > > serves a
> > > > > > > 'Hello, Zest' response from http://localhost:8080/
> > > > > > >
> > > > > > >
> > > > > > > WDYT?
> > > > > > >
> > > > > > > Cheers
> > > > > > > --
> > > > > > > Niclas Hedhman, Software Developer
> > > > > > > http://zest.apache.org/qi4j <http://www.qi4j.org>
- New Energy
> > for
> > > > > Java
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Niclas Hedhman, Software Developer
> > > > > http://zest.apache.org/qi4j <http://www.qi4j.org> - New Energy
for
> > > Java
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Niclas Hedhman, Software Developer
> > http://zest.apache.org/qi4j <http://www.qi4j.org> - New Energy for Java
> >
>



-- 
Niclas Hedhman, Software Developer
http://zest.apache.org/qi4j <http://www.qi4j.org> - New Energy for Java

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message