accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Medinets <david.medin...@gmail.com>
Subject Re: Stream fed accumulo
Date Thu, 10 Apr 2014 12:35:59 GMT
I have stored large numbers of large files in Accumulo using a derivative
of the File System Archive (
https://accumulo.apache.org/1.4/examples/dirlist.html). I had code that
accepted streams but stores chunks instead of the whole file into the
Value. Attached to my ColumnQualifier was essentially a chunk index.


On Thu, Apr 10, 2014 at 7:10 AM, pdread <paul.read@siginttech.com> wrote:

> Hi
>
> This has been bothering me for some time, and I suspect its a dumb
> question,
> but what the heck.
>
> The accumulo client API only accepts byte[] or Text as its Mutation input.
> Would it be possible to
> use a Stream instead (devlopers?)? If I'm processing streams, which I am,
> and I have to handle files to the tune
> of 10GB, which I would like to store in Accumulo but I have read I cannot,
> it would save memory
> footprint on my tomcats if I could stream my data into accumulo and not
> deal
> with bytes/text.
>
> Oh and accumulo developers while you're at adding this new feature it would
> be nice if the bulk loads could
> append instead of just replace the tables....would be nice.
>
> Thanks
>
> Paul
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981.html
> Sent from the Users mailing list archive at Nabble.com.
>

Mime
View raw message