chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <>
Subject [jira] Commented: (CHUKWA-30) Remove HDFS flush & connection holding (Collector)
Date Thu, 14 May 2009 17:59:45 GMT


Jerome Boulon commented on CHUKWA-30:

Theoretically, yes, in practice it's not currently possible.
I have access o the FSOutputStream so I can call flush and it should have just flush to disk
except that the HDFS implementation is not doing anything on the flush instead there's a flushbuffer
method that write to disk but that one is private or protected ...

I gonna see with the HDFS team if there's a way to flush without too much overhead because
the flushbuffer also compute the crc. If that's possible then we can extends the class or
play with introspection to get access to this method ... but yes, I want to be able to flush
after every post so I'll work on that direction.

> Remove HDFS flush & connection holding (Collector)
> --------------------------------------------------
>                 Key: CHUKWA-30
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: data collection
>            Reporter: Jerome Boulon
>            Assignee: Jerome Boulon
>         Attachments: CHUKWA-30.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message