sling-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Egli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SLING-6745) kafka-based sling.event.api implementation
Date Thu, 30 Mar 2017 13:19:41 GMT

    [ https://issues.apache.org/jira/browse/SLING-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949040#comment-15949040
] 

Stefan Egli commented on SLING-6745:
------------------------------------

[~ianeboston], the MoM-based Job API has definitely been considered, as a kafka variant based
on that is pretty straight-forward. However there's the backwards-compatibility argument that
came up: there are many existing users of the Sling Event based Job API which would have to
be ported to the Mom-based Job API - and the aim is to avoid requiring user adjuments as well
as having 2 different APIs in the same deployment.

This is why the suggestion was with SLING-6739 to split sling.event into sling.event.api and
sling.event.resource (in a way that users of the API wouldn't notice anything, ie code compatible).
This would then open the way for other implementations of sling.event.api - such as this kafka
variant.

Now with regards to this new kafka-based implementation and the MoM-based API & implementation
I see the following different variants:
# A bridge implementation could map sling.event.api to [sling.job|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/jobs/core]
and fill in the gaps. One such gap is as mentioned the support for querying for jobs - which
is what you also pointed out. That could be added to this bridge. The implementation of the
kafka part would then be based on the sling.mom SPI.
# An implementation that uses the [MoM API|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/api]
for the messaging part, but doesn't use the [MoM-based Sling Jobs API|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/jobs/core]).
Additionally it would add the job query part, same as above.
# just reuse (ie copy + modify) code from the MoM-based implementation (not the API) and marry
that with the job query part. Then implement everything for Kafka.
# same as 3. but define a different MoM API.

I'm not sure yet which is the best option, but 1) a mix of sling.event.api and sling.job API
(the [org.apache.sling.jobs|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/jobs/core/src/main/java/org/apache/sling/jobs]
package) seems less favorable to me. Whether 2. is possible needs to be seen. 

As a summary : I think it doesn't make sense to reuse the [MoM-based Jobs API|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/jobs/core]
- but it might make sense to use the [MoM API/SPI|https://github.com/apache/sling/tree/trunk/contrib/commons/mom/api].

Wdyt?

> kafka-based sling.event.api implementation
> ------------------------------------------
>
>                 Key: SLING-6745
>                 URL: https://issues.apache.org/jira/browse/SLING-6745
>             Project: Sling
>          Issue Type: New Feature
>            Reporter: Stefan Egli
>            Assignee: Stefan Egli
>
> In job handling to scale for larger deployment it is essential to be able to execute
a job outside of the local instance. This can be in another instance in the same cluster (ie
when ontop of documentMk) or outside of the cluster (in AEM eg via [offloading|https://docs.adobe.com/docs/en/aem/6-2/deploy/configuring/offloading.html]).
In both cases Sling Event (Resource) stores the job in the repository (/var/eventing/jobs).
> It could be useful to have another variant of job execution that is managed entirely
outside of the repository. With SLING-5645 a new API was created to support messaging-based
implementations that would fit in this category as they map a job to a (one or a few) message(s).

> This new SLING-5645 Job-API is not entirely compatible with sling.event.api though.
> This ticket is to track an effort to provide a messaging-based implementation for sling.event.api
- namely for Kafka. The goal is to become a drop-in replacement of sling event without much
need for change on the user side of the sling.event.api.
> This might not right away become part of sling, but might start off in the contrib section
- to underscore that it's not something supported, at least as of yet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message