bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Istvan Soos <istvan.s...@gmail.com>
Subject Re: log compaction of entries
Date Tue, 07 Nov 2017 20:18:30 GMT
On Tue, Nov 7, 2017 at 8:09 PM, Sijie Guo <guosijie@gmail.com> wrote:
> But I would to learn more about your use case and to see how we can support
> you.

It is a nice feature in Kafka, and I've seen a complex app using it:
https://kafka.apache.org/documentation.html#compaction

My use case is really simple: a website is crawled in regular periods,
and it is easy to create a content identifier out of it. I would store
the different versions of it in a ledger for downstream processing,
but there is really no need to preserve all of the versions of the
past if the content identifier is the same.

I know this specific case can be handled in many different ways, but
if the ledger could do that on its own, it could simplify the overall
architecture (free GC).

Istvan

Mime
View raw message