incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <>
Subject Re: Embedding Chukwa agent/adapter?
Date Thu, 08 Apr 2010 00:43:56 GMT
Nothing unreasonable about that modified adaptor.  However, you
presumably want to delete the file, not after sending the chunk, but
after getting back the Ack.  You can implement the interface
"NotifyOnCommitAdaptor" to get ack notifications.


On Wed, Apr 7, 2010 at 2:51 PM, Kirk True <> wrote:
> Hi Ari,
> Question below...
> Ariel Rabkin wrote:
> If the above is correct, then my next question is - how would I know when I
> can reclaim the files on the local storage? How do I know when they've been
> sent to the collector and thus can be deleted from local storage so as to
> prevent unbounded growth of the files?
> If you connect to the agent and say "list", you can see how  much data
> from each adaptor has been committed. If it matches the length of your
> file, you can safely deregister the adaptor and delete the file.
> Using 0.4.0, I never saw -- via the list command -- the file size update.
> The file adapter info would be listed and then simply disappear once the
> file had been transferred. The files are typically less than 1k, so it takes
> very little time to transfer.
> What I ended up doing -- which requires no polling from my application --
> was to extend FileAdaptor with a new class named DeleteOnSuccessFileAdaptor
> which after sending the chunk, deletes the file. I think that approaches
> muddles the separation of concerns, but it appears to work correctly (in
> prototype stage).
> Thoughts?
> Thanks,
> Kirk

Ari Rabkin
UC Berkeley Computer Science Department

View raw message