jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothée Maret <timothee.ma...@gmail.com>
Subject Re: [FileVault][discuss] performance improvement proposal
Date Tue, 07 Mar 2017 08:53:06 GMT
Hi,

2017-03-06 20:52 GMT+01:00 Felix Meschberger <fmeschbe@adobe.com>:

> Hi
>
> This looks great.
>
> As for configuration: What is the reason for having a configuration option
> ? Not being able to decide ? Or real customer need for having it
> configurable ?
>

Setting the compression level is a tradeoff between the compression speed
and the size of the compressed artefacts.

IMO, different use cases favour maximising either of the two or keep the
current default which is a compromise between the two.

For instance, as I see it, Sling Content Distribution would maximise
compression speed and the AEM Quickstart would maximise compression size of
its content packages.

This, IMO, it makes sense to allow configuring/specifying the compression
levels per use case (not globally).


>
> I think we should start with reasonble heuristics first and consider
> configuration options in case there is a need/desire.
>

I have opened JCRVLT-163 to track this. We could indeed add the
configuration later, assuming the increased package size (expected to be <
5% for packages containing already compressed binaries, 0% for other
packages) is not an issue even with size sensitive use cases (such as the
AEM Quickstart).

Regards,

Timothee


>
> Regards
> Felix
>
> Am 06.03.2017 um 16:43 schrieb Timothée Maret <timothee.maret@gmail.com>:
>
> Hi,
>
> With Sling content distribution (using FileVault), we observe a
> significantly lower throughput for content packages containing binaries.
> The main bottleneck seems to be the compression algorithm applied to every
> element contained in the content package.
>
> I think that we could improve the throughput significantly, simply by
> avoiding to re-compress binaries that are already compressed.
> In order to figure out what binaries are already compressed, we could use
> match the content type stored along the binary against a list of
> configurable content types.
>
> I have done some micro tests with this idea (patch in [0]). I think that
> the results are promising.
>
> Exporting a single 250 MB JPEG is 80% faster (22.4 sec -> 4.3 sec) for a
> 3% bigger content package (233.2 MB -> 240.4 MB)
> Exporting AEM OOTB /content/dam is 50% faster (11.9 sec -> 5.9 sec) for a
> 5% bigger content package (92.8 MB -> 97.4 MB)
> Import for the same cases is 66% faster respectively 32% faster.
>
> I think this could either be done by default and allowing to configure the
> list of types that skip compression.
> Alternatively, it could be done on a project level, by extending FileVault
> with the following
>
> 1. For each package, allow to define the default compression level (best
> compression, best speed)
> 2. Expose an API that allow to plugin a custom logic to decide how to
> compress a given artefact
>
> In any case, the changes would be backward compatible. Content packages
> created with the new code would be installable on instances running the old
> code and vice versa.
>
> wdyt ?
>
> Regards,
>
> Timothee
>
>
> [0] https://github.com/tmaret/jackrabbit-filevault/tree/
> performance-avoid-compressing-already-compressed-binaries-
> based-on-content-type-detection
> [1] https://docs.oracle.com/javase/7/docs/api/java/util/
> zip/Deflater.html#BEST_SPEED
>
>

Mime
View raw message