sling-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Chopra (Jira)" <>
Subject [jira] [Commented] (SLING-8854) In-file and In-memory queue-providers for Forward Distribution report incorrect value for processing "Attempts"
Date Fri, 22 Nov 2019 12:21:00 GMT


Ashish Chopra commented on SLING-8854:

One way to address this would be to add a capability for recording processing attempts in
the {{DistributionQueue}} SPI.
Since it is entirely up to the implementation of the {{DistributionQueue}} to decide what
to do when processing attempt is recorded, a {{default}} implementation of a no-op can be
added to the SPI so that existing clients don't break.

_Assumption:_ Java  < 8 are not expected to be supported anymore (esp. given Oracle itself
has dropped the ball on it).

Patch at  [^0001-SLING-8854-In-file-and-In-memory-queue-providers-for.patch] illustrates the
approach by:
* adding {{DistributionQueue.recordProcessingAttempt}} method
* implementing {{SimpleDistributionQueue.recordProcessingAttempt}}
* updating {{}} to call the added method while dequeing
the item from queue-head.

If the approach is deemed acceptable, I will work on adding tests and raising a PR.

[~marett], would be great if you can provide your initial feedback on this.

> In-file and In-memory queue-providers for Forward Distribution report incorrect value
for processing "Attempts"
> ---------------------------------------------------------------------------------------------------------------
>                 Key: SLING-8854
>                 URL:
>             Project: Sling
>          Issue Type: Bug
>          Components: Content Distribution
>    Affects Versions: Content Distribution Core 0.4.0
>            Reporter: Ashish Chopra
>            Priority: Major
>         Attachments: 0001-SLING-8854-In-file-and-In-memory-queue-providers-for.patch
> {{ForwardDistributionAgentFactory}} allows specifying "In-Memory" and "In-File" queue-providers
[0]. Both these implementations are provided by {{SimpleDistributionQueue}} [1].
> While responding to a queue-processing request, the head of the queue is fetched to create
a {{DistributionQueueEntry}}, which encapsulates a {{DistributionQueueItemStatus}}. While
retrieving the queue-head-entry, the number-of-attempts are also incremented by 1, implying
_one more_ processing-attempt to have happened on a given queue-item. [2]
> The trouble arises from the fact that {{SimpleDistributionQueue#getHead}} ends up being
called from at least three other places:
> # Queue status Health Check [3], which is a periodic job
> # UI (via {{ExtendedDistributionServiceResourceProvider}}) [4]
> # Distribution Queue status MBean [5]
> Out of these, #1 is most troublesome - because of its periodic nature, the number of
Attempts for a queue-item that's being processed (or, is blocked) will keep on increasing
without corresponding dequeue-attempts.
> Due to #2, everytime the queue status is delivered to the UI, the attempt would again
increase. Similar would be the case of MBean invocations.
> This needs to be addressed. Notably, "Sling Jobs" based queue doesn't suffer from this
issue because it relies on retry-counts as maintained by Sling Job Execution Engine.
> [0]
> [1]
> [2]
> [3]
> [4]
> [5]

This message was sent by Atlassian Jira

View raw message