fineract-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Woodlock <keithwoodl...@gmail.com>
Subject Re: Microservices
Date Tue, 24 May 2016 09:23:56 GMT
Myrle,

Sounds like Markus/you have been on this journey for a while already.
Everything sounds great. Be very interested in seeing it come together.

For that reason I think the sooner you are transparent about where the code
is the better so the community can take a look also and help out where
possible.

regards,
Keith.



On Tue, May 24, 2016 at 9:07 AM, Myrle Krantz <mkrantz@mifos.org> wrote:

> Hey Keith,
>
> I’ll start with the easy questions:
>
> * Have I done this before? Yes.  But no woman is an island.  I’m happy to
> have qualified advice.
>
> * Where will Markus and I be saving our work? We’ll send you a pointer once
> we feel it would be productive for others to look at it.  Until we have a
> common understanding of the design and technologies sharing code would be
> fruitless.
>
> I thought about your suggestion for a general approach.  I agree we should
> start with a limited scope, and I like your component break down.  I would
> refine that as follows:  Since it’s always been important that
> Mifos/Fineract is multi-tenanted, I think the best place to start is with a
> service for tenant allocation and service orchestration.  From there, a
> service for user/permission provisioning would be the obvious next step.
> Together these represent your service 1.  Splitting them is not a
> fundamentally different design.  It is cleaner though, since the user
> management would be within a tenant, and the only service that should be
> “tenant-aware” is the one that does tenant allocation.  For your second
> service, you suggest attacking loans next.  Savings is just as important as
> loans, and fundamentally equivalent (just black numbers versus red ones).
> So after tenant allocation and user creation were complete, the next
> bare-bones service should be a ledger.  Savings and loans products would
> then be part of a fourth service for describing product offerings.  Again,
> I’ve taken what you represented as one service and made two out of it.
>
> All the services except the tenant creation service should treat all data
> addition/adjustment as a command, just like you originally designed it for
> Mifos.  That approach has worked well and we should stick to it.  But
> command processing should be asynchronous for *everything*.  High volumes
> of write operations in *any* service should not throttle capacity for any
> other service.  The tenant creation service is the one service that can’t
> use the command pattern, because commands are serialized to a database the
> tenant creation service provisions.
>
> These are the components I just described:
>
> A.) A command module for writing and handling commands which would be used
> by all services.  This is a compile time dependency.
>
> B.) A tenant service which creates the databases needed for the other
> services, plus the tables needed for command persistence.
>
> C.) A user service which creates the users in the system.
>
> D.) A ledger service which tracks payments.  Depends on the command module,
> the tenant service, and the user service.
>
> E.) A products service which can be used to configure savings and loans
> products.  Depends on all the other components I described.
>
> B & C = Keith’s 1.)
>
> D & E = Keith’s 2.)
>
>
> As we move forward we'll probably break it down further than that.
>
> These services should be deployed in jetty or tomcat much as today’s Mifos
> is.  We can use Eureka and Ribbon for service discovery if we are deploying
> multiple instances of a given service.  I’ve also really liked what I’ve
> seen of docker.  I would like to ultimately build a docker image for each
> service, and make the build scripts for those docker images part of the
> service projects.
>
> This leaves me with a question for a mentor: As far as I can tell the
> current mode of operation at Apache is one repository to one product.  I
> would prefer to work with one repository per service.  I believe that would
> help programmers remain strict about division of labor between the
> services, and think more carefully about interface breaking changes.  Is
> there any reason a product can’t have multiple repositories?
>
> Lest I create a false impression, this is not my design alone.  Markus
> created most of it.  I’ve been refining it together with him.
>
>
> Greets from the Voreifel, Germany
>
> Myrle
>
>
>
>
> *Myrle Krantz*
> Solutions Architect
> RɅĐɅЯ, The Mifos Initiative
> mkrantz@mifos.org | Skype: mkrantz.mifos.org | http://mifos.org
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>
>
> On Mon, May 23, 2016 at 11:16 AM, Keith Woodlock <keithwoodlock@gmail.com>
> wrote:
>
> > Mryle,
> >
> > >> Now please excuse me while I get back to just doing it. ; o)
> >
> > Ok sounds like decision made that yourself and Markus will be going ahead
> > with this :)
> >
> > Can you let us know where you will be checking in the code so we in the
> > community can follow along.
> >
> > Given your are going to go ahead with developing a proof of concept
> > starting from a blank canvas it might be good to layout a scope and some
> > goals to get to in a time-boxed period of time. I would suggest something
> > like:
> >
> > 1) A single microservice implemented from API to Database including
> > deployment related capabilities
> >
> > The emphasis on this would be to show off the tech approach/pattern
> > involved in implementing a microservice over the domain of the
> microservice
> > so feel free to choose a simpler area
> >
> > This hopefully could be provided in a much shorter time-line (a month or
> > so)
> >
> > 2) A microservice that encapsulates the full Loan Product / Loan area of
> > Fineract
> >
> > Makes sense to takcle the meat of the platform to show off qualities of
> > microservices approach
> >
> > By the end of this you will have two microservice examples and that will
> > bring up the questions around deployment and service discovery and how
> that
> > will be implemented.
> >
> > One last question would be to ask if Markus or yourself have actually
> > developed/being part of a project previously that has used the
> microservice
> > approach specifically you are talking about?
> >
> > If yes thats great, if no, is it worth seeking someone with some
> expertise
> > in the area of microservices specifically in java area?
> >
> > regards,
> > Keith.
> >
> >
> > On Mon, May 23, 2016 at 8:26 AM, Myrle Krantz <mkrantz@mifos.org> wrote:
> >
> > > Hi,
> > >
> > > I won't be able to write a complete answer.  Nobody would read an
> e-mail
> > > that long.  So I’m going to “nibble” at it bit by bit.
> > >
> > > Of the three approaches Keith lists, I favor the third approach
> > "Re-imagine
> > > the fineract platform from ground up again (hope it solves the problems
> > you
> > > see and doesnt introduce new problems)".  I've been in the industry
> long
> > > enough (16 years) to understand that there are significant risks in
> > > "starting over from scratch", and I hope that we can learn enough from
> > the
> > > existing code base to avoid some of those road bumps.
> > >
> > >
> > > I would imagine multiple phases here:
> > >
> > > 1.) Markus and I start work on a framework, and basic services for this
> > > refactoring.  Everyone else continues work on MifosX
> > >
> > > 2.) We do a proof of concept with a bare bones implementation.
> > >
> > > 3.) Report to the community based on what we learn by end of this year.
> > >
> > > 4.) Based on what we have learned, together with the community consider
> > > migration of:
> > >
> > >    a.) New microservice-based code into the Fineract community.
> > >
> > >    b.) Community man-power from the existing codebase to the new code
> > base.
> > >
> > >    c.) Existing customer data into the new microservice-based code via
> > > migration scripts.
> > >
> > >    d.) Existing projects onto the new microservice-based product.
> > >
> > >    e.) The community app onto the new microservice-based backend.
> > >
> > > We can do a subset of the above migration steps.  There are several
> > > possible approaches.  But I really want to do this together with the
> > > Fineract community.  There’s much know-how here, and I think we can
> copy
> > > code over wholesale in many areas, and in other areas plagiarize good
> > > architectural decisions and problem-solving approaches which the
> existing
> > > Fineract codebase illustrates.
> > >
> > > Because I want to do this with the Fineract community, I want to start
> > > getting people in the community up to speed on the concepts early.
> Even
> > if
> > > people decide they don't like this approach, nobody will be poorer for
> > the
> > > extra information.  I also very much want people’s feedback on where
> they
> > > see problems with our approach. I haven’t given much in the way of
> > > specifics yet; I want people to have some background first. Which was
> the
> > > reason for my original mail.
> > >
> > > Now please excuse me while I get back to just doing it. ; o)
> > >
> > > (My next email will come tomorrow, and will focus on Roman’s very
> > important
> > > deployment question.)
> > >
> > > Greets from the Voreifel, Germany,
> > > Myrle
> > >
> > >
> > > *Myrle Krantz*
> > > Solutions Architect
> > > RɅĐɅЯ, The Mifos Initiative
> > > mkrantz@mifos.org | Skype: mkrantz.mifos.org | http://mifos.org
> > > <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> > >
> > >
> > > On Thu, May 19, 2016 at 11:49 PM, Keith Woodlock <
> > keithwoodlock@gmail.com>
> > > wrote:
> > >
> > > > Hi Mryle,
> > > >
> > > > Thanks for your detailed reply, I added my comments (please remember
> I
> > am
> > > > not involved in code base for sometime so might be missing alot)
> below
> > > but
> > > > its a pretty long email so I summarise what I think here:
> > > >
> > > > The decision to move from one approach to developing a system to
> > another
> > > is
> > > > a big one and not without its costs. I am not really for or against
> it
> > > and
> > > > I'll be interested in seeing how things turn out whichever way the
> > > > community decides to go. If everyone really feels it will be the
> > solution
> > > > to any problems that exist then they should go for it. The risk of
> > course
> > > > is that its a very tech solution to things.
> > > >
> > > > I think Chris Richardson does a good job on http://microservices.io
> of
> > > > outlining the differences between the monolith (
> > > > http://microservices.io/patterns/monolithic.html) and the
> > microservices
> > > (
> > > > http://microservices.io/patterns/microservices.html) approaches.
> > > >
> > > > You probably have the following 3 directions you can go:
> > > >
> > > > 1) Stay with the monolith and attempt to tackle the problems you see
> > with
> > > > the system (domain, performance etc)
> > > >
> > > > 2) Gradually move services from the monolith into microservices (hope
> > it
> > > > solves the problems you see and doesnt introduce new problems)
> > > >
> > > > 3) Re-imagine the fineract platform from ground up again (hope it
> > solves
> > > > the problems you see and doesnt introduce new problems)
> > > >
> > > > - Approach 2 sounds like it will be a drawn out affair and take a
> long
> > > time
> > > > to reach the end with the drawback of having to support both
> approaches
> > > >
> > > > - Approach 3 it too would be alot of effort but you could use
> something
> > > > like http://eventuate.io/ to get to a poof of concept faster
> > > >
> > > >
> > > >
> > > >
> > > > Before devling into some of the points you raised I wanted to
> highlight
> > > two
> > > > aspects of the fineract platform:
> > > >
> > > > *- Everything is exposed a HTTP API / Service*
> > > >
> > > > All the functionality that exists in the platform is or should be
> > exposed
> > > > by a HTTP API. This HTTP API is a techonology agonistic contract that
> > > > developers can build their apps/clients on or integrate with easily.
> > > >
> > > > Developers can choose to build their apps/clients in any technology
> > they
> > > > wish and to use as much or as little of the platform API in their
> app.
> > > >
> > > > API documented here https://demo.openmf.org/api-docs/apiLive.htm
> > > >
> > > > *- All HTTP API Endpoints deployed together*
> > > >
> > > > The code is compiled and bundled as a .war file that is deployed to
> an
> > > > application server like tomcat.
> > > >
> > > > If one application server isnt able to handle the request load you
> can
> > > > easily scale horizontally by adding more application servers (as the
> > > > platform is stateless) and load balance the requests between the
> > > serrvers.
> > > > This would typically only be needed in a larger scale deployment.
> > > >
> > > >
> > > > *The Problems You Listed*
> > > >
> > > >
> > > >
> > > >
> > > > *1.) We are starting to have performance problems.  Conflux in
> > particular
> > > > is running into this.  Mussoni is starting to have problems too.  All
> > of
> > > us
> > > > would like to move to bigger customers and bigger markets, so this
> will
> > > > only get worse.*
> > > >
> > > > You should open up a new email thread within the community on these
> > > issues.
> > > >
> > > > It would be useful to provide some more details on what API requests
> in
> > > > praticular Conflux/Musoni and others are seeing performance problems
> > > with.
> > > >
> > > >
> > > > *2.) As we move to bigger customers, down-time becomes less and less
> > > > acceptable.*
> > > >
> > > > There shouldnt need to be a need for downtime with the existing
> > > > deploy-all-services-as-one approach.
> > > >
> > > > On tomcat you can deploy your new application in parallel with your
> old
> > > > one.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *3.) Loans products are hard-coded.  A lot of dev list mails read
> like
> > > this
> > > > “For my case, I want my interest/fees calculated at a different time/
> > on
> > > a
> > > > different basis than the ‘standard’ use case, how do I do this?”
 If
> I
> > > were
> > > > a customer asking a question like this, some of the hacks being
> > suggested
> > > > would feel brittle and unsatisfying.  Savings products look like they
> > > were
> > > > implemented as an afterthought and are also hard-coded.*
> > > >
> > > > Sounds likes a domain modelling / requirements issue or an
> > implementation
> > > > issue.
> > > >
> > > > Different financial institutions in different countries have many
> > > different
> > > > flavours of the way they want their loan or saving products to work
> and
> > > its
> > > > an area that will continue to change into the future.
> > > >
> > > > Mifos started out with supporting one or two of the common models in
> > use
> > > in
> > > > the field and it looks like more have been added in since based on
> > > content
> > > > in http://fineract.incubator.apache.org/
> > > >
> > > > You might be suggesting though that its time to revisit the Loan
> > Product
> > > /
> > > > Loan area and re-model it based on what you know today?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *4.) Our adherence to the CQRS pattern is less than perfect.  Command
> > > > responses were hard coded.  Some work has been done to make proper
> > > command
> > > > handlers out of it, but there are still semi-commands handled in a
> big
> > if
> > > > else block.  Handling commands is synchronous even though one of the
> > > > advantages of this pattern should be the ability to handle operations
> > > > asynchronously.  This contributes to the problems mentioned in 1
> > above.*
> > > >
> > > > Not sure what the hard-coded command response area is (been along
> time
> > > > since looked at code base)
> > > >
> > > > Its true that code that is responsible for identifying what command
> > > handler
> > > > to invoke based on the command is not great and suffered from the
> 'god
> > > > class' problems - the inital core of the platform was written in a
> > > > time-boxed three month period as proof of concept and as more
> > > apis/commands
> > > > got added it got bigger.
> > > >
> > > > That said, if you have a preferred solution its surely an area that
> > could
> > > > of been improved in the last few years if it was that important?
> > > >
> > > > On the synchronous nature of them, yes we decided rather than going
> for
> > > an
> > > > eventually-consistent system that the api calls would be like a
> > > datatabase
> > > > transaction call, i.e when you get a response you know its either
> > > > successful or not
> > > >
> > > > Calling an api 1000's of times to do something in bulk of course
> would
> > > not
> > > > make much sense and you would expect to add a bulk api for the that
> > type
> > > of
> > > > activity.
> > > >
> > > > Also if there are very long running tasks then of course they should
> > not
> > > be
> > > > done through a synchronous api call, you should set up an api that
> > > supports
> > > > 'long running tasks' and pass back a id/key that the user can use to
> > > check
> > > > for the result later or something similiar.
> > > >
> > > >
> > > >
> > > >
> > > > *5.) There’s currently no standard way to connect to what I like to
> > call
> > > > “money buses” (Swift, Stellar, UPI).  I did implement a bridge for
> > > Stellar
> > > > which listened for commands, but it wasn’t clear which commands to
> > listen
> > > > for.  I ended up deciding on journal entries.*
> > > >
> > > > Not fully across what you are doing here but if its a case of
> > integrating
> > > > some kind of third party payments api and invoking it based on some
> api
> > > > call like makePayment or disburse then that should be catered for by
> > the
> > > > platform. If you werent clear on how to do its probably cause the
> > > platform
> > > > didnt try to support that type of integration out of the box first of
> > > all.
> > > >
> > > > Listening to journal entries might not be the best idea cause what
> If i
> > > > dont want to use the platforms accounting solution and turn it off?
> > > >
> > > > This use case should be designed into the praticular area in
> question.
> > > >
> > > >
> > > >
> > > >
> > > > *6.) In first world core banking systems, the back end and the front
> > end
> > > > are carefully separated with clearly defined and limited
> communication.
> > > > This is to provide added protection against hacking of the type we’ve
> > > seen
> > > > in the news recently (Bangladesh’s central bank for example).*
> > > >
> > > > The front end (apps/clients) are totaly seperated from the back-end
> api
> > > at
> > > > present. In terms of limiting what apps can speak to what APIs, you
> > > > probably need to add that capability on top of the current
> > > > authentication/authorization capablity.
> > > >
> > > > At present there is fine-grained support for restricting what apis a
> > user
> > > > is entitled to call, its not to much more to also limit what apis can
> > be
> > > > called based on what app/client is being used.
> > > >
> > > > *How these things related to microservices pionts*
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *1.) By chopping up database accesses into logical units, we can
> > increase
> > > > capacity for only those units which need it.  This reduces costs.  In
> > > > particular, for the saving and reading of commands (for which we
> expect
> > > the
> > > > largest volume of write instructions), we can switch to a faster
> > > > persistence mechanism which works better in a distributed
> environment.*
> > > >
> > > > Your probably talking about switching to storing the commands/events
> > in a
> > > > different persistence mechanism than a relational database. You could
> > > > choose to do that now if you wish.
> > > >
> > > > You might also be referring to breaking the current database schema
> > into
> > > a
> > > > database per service approach (
> > > http://microservices.io/patterns/index.html
> > > > ).
> > > >
> > > >
> > > > I am not sure how this would reduce costs. Like any solution it has
> its
> > > > pros and cons (
> > > > http://microservices.io/patterns/data/database-per-service.html)
> > > >
> > > >
> > > >
> > > > *2.) We can run multiple instances of a microservice, and, with
> careful
> > > > design, perform rolling updates, moving us towards the ability to
> have
> > > > 99.99% uptime.*
> > > >
> > > > Much like point 2 earlier,  you should be able to do this with the
> > > current
> > > > setup.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *3.) A more modular approach with well-defined APIs would make it
> > > possible
> > > > among other things to separate out the loans and savings products and
> > let
> > > > anyone implement their own loans and savings products who wants to.
> > > Things
> > > > like interest rates, fees, and repayments rates are very cultural,
> and
> > as
> > > > we move into more countries we’ll need more flexibility here.*
> > > >
> > > > There should be well defined APIs at present. If you have a solution
> > for
> > > > allowing people to create their own loans and savings products with
> > > > complete flexibiliy you should be able to roll it out now in the
> > current
> > > > setup.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *4.) CQRS is a truly awesome pattern for this domain, partly because
> it
> > > in
> > > > theory, makes it possible to handle writes asynchronously.  If we can
> > > > decouple the commands for different domains from each other and
> > actually
> > > > handle them asynchronously that would be a major step forward.  Of
> > course
> > > > this doesn’t necessitate a microservices deployment model, at least
> not
> > > > until the question of what we do with the data we create while
> handling
> > > the
> > > > commands.  If, for example, a command for creating a user causes
> > > different
> > > > data to be adjusted then a command for starting a payment, why should
> > > > creating a user cause a slowdown in working through payments?  Those
> > two
> > > > kinds of processing don’t even have to execute on the same
> hardware...*
> > > >
> > > > As you say, if you want to implement a more fully blown
> implementation
> > of
> > > > the CQRS+Event Sourcing pattern you could implement that now for the
> > > areas
> > > > your believe need it and still respect the existing API
> > > >
> > > > The example you given isnt a good one as you wouldnt expect there to
> > be a
> > > > high throughput of users created but I understand that your saying
> that
> > > if
> > > > there was 'intensive' operations going on on the platform, it might
> be
> > > > usefult to be able to split them into a seperate unit that runs on
> > > seperate
> > > > hardware so it doesnt impact more important functionality that might
> > have
> > > > an SLA on it like payment processing etc
> > > >
> > > > The question is, what are these intensive operations that might be
> > > > affecting other areas of the platform so badly?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *5.) Writing ‘money bus’ plugins as microservices is a great way to
> > > > separate functionality and be sure that even if one bus is down, the
> > > others
> > > > can still be up and running.  My stellar bridge component essentially
> > > > applied this pattern.  If we can standardize the interface that
> bridge
> > > > components implement, we could make these easier to write and improve
> > > Mifos
> > > > connectivity to other systems.  These don’t need to be a standard
> part
> > of
> > > > deployment, in different countries, different networks are popular,
> and
> > > > these things change over time too.  Connectivity to other systems is,
> > in
> > > my
> > > > humble opinion, one of the most important factors in the success of a
> > > > product.*
> > > >
> > > > Yes its very important to be able to integrate with other systems
> and I
> > > > dont see why the kind of functionality you want to implement and
> > > integrate
> > > > into mifos/fineract cant be done today in a clean way. I dont see how
> > > > having all the current services split out into deployable units helps
> > > this
> > > > use case.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *6.) Separation of core systems from customer-visible systems will
> > > require
> > > > a duplication of some functionality.  This functionality would need
> to
> > > run
> > > > in separate processes and only have access to data duplicated out of
> > the
> > > > back end system.  Most data would only flow from back to front
> > (probably
> > > > via some kind of ETL process).  If we divvy up our data based on the
> > > > functionality groups which need it, it will be much easier to
> determine
> > > > which data should be copied “forward” and which should “stay behind”.
> > A
> > > > microservice is such a functionality group.*
> > > >
> > > > Its probably not a good idea to mix customer-visble functionality
> with
> > > core
> > > > business-functionality. It terms of large financial institutions you
> > > would
> > > > be talking about 100,000s to millions of potentional users for
> customer
> > > > visible functionality whilst the core business-functionality would be
> > > > significantly less.
> > > >
> > > > So they really are two different systems at some level.
> > > >
> > > > regards,
> > > > Keith.
> > > >
> > > >
> > > > On Thu, May 19, 2016 at 2:07 PM, Myrle Krantz <mkrantz@mifos.org>
> > wrote:
> > > >
> > > > > This is a big topic, and I’m trying to decide if I should chop
it
> up
> > > into
> > > > > multiple threads.  I’m going to leave it as one for now, since
I
> > > detect a
> > > > > healthy scepticism about the question of whether we need to do this
> > at
> > > > all,
> > > > > and I don’t want to preclude that discussion.  Here are some of
the
> > > > > problems I’m seeing:
> > > > >
> > > > > 1.) We are starting to have performance problems.  Conflux in
> > > particular
> > > > is
> > > > > running into this.  Mussoni is starting to have problems too.  All
> of
> > > us
> > > > > would like to move to bigger customers and bigger markets, so this
> > will
> > > > > only get worse.
> > > > > 2.) As we move to bigger customers, down-time becomes less and less
> > > > > acceptable.
> > > > > 3.) Loans products are hard-coded.  A lot of dev list mails read
> like
> > > > this
> > > > > “For my case, I want my interest/fees calculated at a different
> time/
> > > on
> > > > a
> > > > > different basis than the ‘standard’ use case, how do I do this?”
> If
> > I
> > > > were
> > > > > a customer asking a question like this, some of the hacks being
> > > suggested
> > > > > would feel brittle and unsatisfying.  Savings products look like
> they
> > > > were
> > > > > implemented as an afterthought and are also hard-coded.
> > > > > 4.) Our adherence to the CQRS pattern is less than perfect.
> Command
> > > > > responses were hard coded.  Some work has been done to make proper
> > > > command
> > > > > handlers out of it, but there are still semi-commands handled in
a
> > big
> > > if
> > > > > else block.  Handling commands is synchronous even though one of
> the
> > > > > advantages of this pattern should be the ability to handle
> operations
> > > > > asynchronously.  This contributes to the problems mentioned in 1
> > above.
> > > > > 5.) There’s currently no standard way to connect to what I like
to
> > call
> > > > > “money buses” (Swift, Stellar, UPI).  I did implement a bridge
for
> > > > Stellar
> > > > > which listened for commands, but it wasn’t clear which commands
to
> > > listen
> > > > > for.  I ended up deciding on journal entries.
> > > > > 6.) In first world core banking systems, the back end and the front
> > end
> > > > are
> > > > > carefully separated with clearly defined and limited communication.
> > > This
> > > > > is to provide added protection against hacking of the type we’ve
> seen
> > > in
> > > > > the news recently (Bangladesh’s central bank for example).
> > > > >
> > > > > So how does this relate to Microservices?  From here on out, I’m
> > going
> > > to
> > > > > assume you all know what a Microservice is, so I hope you read
> those
> > > > links
> > > > > in my original e-mail : o).
> > > > >
> > > > > 1.) By chopping up database accesses into logical units, we can
> > > increase
> > > > > capacity for only those units which need it.  This reduces costs.
> In
> > > > > particular, for the saving and reading of commands (for which we
> > expect
> > > > the
> > > > > largest volume of write instructions), we can switch to a faster
> > > > > persistence mechanism which works better in a distributed
> > environment.
> > > > > 2.) We can run multiple instances of a microservice, and, with
> > careful
> > > > > design, perform rolling updates, moving us towards the ability to
> > have
> > > > > 99.99% uptime.
> > > > > 3.) A more modular approach with well-defined APIs would make it
> > > possible
> > > > > among other things to separate out the loans and savings products
> and
> > > let
> > > > > anyone implement their own loans and savings products who wants to.
> > > > Things
> > > > > like interest rates, fees, and repayments rates are very cultural,
> > and
> > > as
> > > > > we move into more countries we’ll need more flexibility here.
> > > > > 4.) CQRS is a truly awesome pattern for this domain, partly because
> > it
> > > in
> > > > > theory, makes it possible to handle writes asynchronously.  If we
> can
> > > > > decouple the commands for different domains from each other and
> > > actually
> > > > > handle them asynchronously that would be a major step forward.  Of
> > > course
> > > > > this doesn’t necessitate a microservices deployment model, at least
> > not
> > > > > until the question of what we do with the data we create while
> > handling
> > > > the
> > > > > commands.  If, for example, a command for creating a user causes
> > > > different
> > > > > data to be adjusted then a command for starting a payment, why
> should
> > > > > creating a user cause a slowdown in working through payments?
> Those
> > > two
> > > > > kinds of processing don’t even have to execute on the same
> > hardware...
> > > > > 5.) Writing ‘money bus’ plugins as microservices is a great way
to
> > > > separate
> > > > > functionality and be sure that even if one bus is down, the others
> > can
> > > > > still be up and running.  My stellar bridge component essentially
> > > applied
> > > > > this pattern.  If we can standardize the interface that bridge
> > > components
> > > > > implement, we could make these easier to write and improve Mifos
> > > > > connectivity to other systems.  These don’t need to be a standard
> > part
> > > of
> > > > > deployment, in different countries, different networks are popular,
> > and
> > > > > these things change over time too.  Connectivity to other systems
> is,
> > > in
> > > > my
> > > > > humble opinion, one of the most important factors in the success
> of a
> > > > > product.
> > > > > 6.) Separation of core systems from customer-visible systems will
> > > > require a
> > > > > duplication of some functionality.  This functionality would need
> to
> > > run
> > > > in
> > > > > separate processes and only have access to data duplicated out of
> the
> > > > back
> > > > > end system.  Most data would only flow from back to front (probably
> > via
> > > > > some kind of ETL process).  If we divvy up our data based on the
> > > > > functionality groups which need it, it will be much easier to
> > determine
> > > > > which data should be copied “forward” and which should “stay
> behind”.
> > > A
> > > > > microservice is such a functionality group.
> > > > >
> > > > > There are other advantages to microservices that are more general
> in
> > > > nature
> > > > > and not specific to our use case; read the links I sent in my first
> > > > e-mail
> > > > > and you should start to get a feel for it.
> > > > >
> > > > > (Roman, I’m going to watch and see if your questions are answered
> as
> > we
> > > > > discuss.  If not I’ll circle back to them, and see what was
> missed.)
> > > > >
> > > > > Greets,
> > > > > Myrle
> > > > >
> > > > >
> > > > > *Myrle Krantz*
> > > > > Solutions Architect
> > > > > RɅĐɅЯ, The Mifos Initiative
> > > > > mkrantz@mifos.org | Skype: mkrantz.mifos.org | http://mifos.org
> > > > > <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> > > > >
> > > > >
> > > > > On Thu, May 19, 2016 at 1:01 AM, Keith Woodlock <
> > > keithwoodlock@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Myrle,
> > > > > >
> > > > > > First off, thanks for starting a conversation around this within
> > the
> > > > > > fineract dev community. I was wondering when this was going
to
> come
> > > up.
> > > > > >
> > > > > > Rather than delving into the technicalities of what a
> microservice
> > is
> > > > and
> > > > > > isnt I thought it would be good if the people proposing this
> could
> > go
> > > > > into
> > > > > > a bit of detail here of why they are proposing it, the pros
and
> > cons
> > > > of a
> > > > > > microservice approach over the existing approach and the existing
> > > > > problems
> > > > > > within fineract/mifos that it will help solve/remove.
> > > > > >
> > > > > > I think that could promote some good discussion and following
on
> > from
> > > > > that
> > > > > > later we could delve into how it could be achieved if going
that
> > > > > direction.
> > > > > >
> > > > > > regards,
> > > > > > Keith.
> > > > > >
> > > > > >
> > > > > > On Wed, May 18, 2016 at 11:14 PM, Roman Shaposhnik <
> > > > roman@shaposhnik.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Myrle,
> > > > > > >
> > > > > > > microservices are indeed a nice architectural pattern with
an
> > > > > > exponentially
> > > > > > > growing adoption in the enterprise. It typically leverages
some
> > > kind
> > > > > of a
> > > > > > > PaaS
> > > > > > > solution to be available for deployment. My current
> understanding
> > > is
> > > > > that
> > > > > > > your
> > > > > > > deployment model today is a traditional appserver. Are
you
> going
> > to
> > > > > > change
> > > > > > > that?
> > > > > > > If so, what will be the deployment requirements for the
> project?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Roman.
> > > > > > >
> > > > > > > On Wed, May 18, 2016 at 2:35 AM, Myrle Krantz <
> mkrantz@mifos.org
> > >
> > > > > wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > As many of you know Markus and I are thinking through
a
> > > > > rearchitecting
> > > > > > of
> > > > > > > > Fineract into microservices. Markus gave some of you
a
> preview
> > > into
> > > > > our
> > > > > > > > work at the Mifos tech conference in Amsterdam in
March
> (those
> > of
> > > > you
> > > > > > who
> > > > > > > > are interested and weren’t able to attend can check
it out on
> > > > Youtube
> > > > > > > here:
> > > > > > > > https://www.youtube.com/watch?v=iIrnZpoNZ9A -- my
apologies
> > for
> > > > the
> > > > > > poor
> > > > > > > > video quality)
> > > > > > > >
> > > > > > > > If you want to get an introductory understanding of
what a
> > > > > microservice
> > > > > > > is,
> > > > > > > > this is a good place to go:
> > > > > > > >
> > > > > > > > http://martinfowler.com/articles/microservices.html
> > > > > > > >
> > > > > > > > If there is interest, (and I have time), I’ll try
to send you
> > > more
> > > > > > > > information about my ideas as they take shape.
> > > > > > > >
> > > > > > > > Greets,
> > > > > > > >
> > > > > > > > Myrle
> > > > > > > >
> > > > > > > > P.S.  Here’s Spring’s take on this:
> > > > > > > > https://spring.io/blog/2015/07/14/microservices-with-spring
> > > > > > > >
> > > > > > > >
> > > > > > > > *Myrle Krantz*
> > > > > > > > Solutions Architect
> > > > > > > > RɅĐɅЯ, The Mifos Initiative
> > > > > > > > mkrantz@mifos.org | Skype: mkrantz.mifos.org |
> > http://mifos.org
> > > > > > > > <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message