hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: local node Quotas (for an R&D cluster)
Date Fri, 25 Sep 2009 15:04:12 GMT
> I have been mulling over just that sort of thing.  Some sort of
> HadoopAppender that outputs files in the SequenceFile format and
> periodically submits to a DFS node.  Hey,   this is something I _can_
> contribute!  I might start another thread.  Thanks.
> Paul

Look at scribe it was decided to handle messages and add them to
hadoop in chunks. Also there is a hive jira that talkes about a
similar feature "Super Highway" I think that is dealing with near real
time <5 minute processing.


View raw message