incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kirk True <k...@mustardgrain.com>
Subject Re: Embedding Chukwa agent/adapter?
Date Wed, 07 Apr 2010 21:51:17 GMT
Hi Ari,

Question below...

Ariel Rabkin wrote:
>> If the above is correct, then my next question is - how would I know when I
>> can reclaim the files on the local storage? How do I know when they've been
>> sent to the collector and thus can be deleted from local storage so as to
>> prevent unbounded growth of the files?
>>     
>
> If you connect to the agent and say "list", you can see how  much data
> from each adaptor has been committed. If it matches the length of your
> file, you can safely deregister the adaptor and delete the file.
>   

Using 0.4.0, I never saw -- via the list command -- the file size 
update. The file adapter info would be listed and then simply disappear 
once the file had been transferred. The files are typically less than 
1k, so it takes very little time to transfer.

What I ended up doing -- which requires no polling from my application 
-- was to extend FileAdaptor with a new class named 
DeleteOnSuccessFileAdaptor which after sending the chunk, deletes the 
file. I think that approaches muddles the separation of concerns, but it 
appears to work correctly (in prototype stage).

Thoughts?

Thanks,
Kirk

Mime
View raw message