accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Gresock <jgres...@gmail.com>
Subject Re: Stream fed accumulo
Date Thu, 10 Apr 2014 11:46:56 GMT
We were able to use this implementation in our code to stream to and from
Accumulo:
https://github.com/calrissian/accumulo-recipes/blob/master/store/blob-store/src/main/java/org/calrissian/accumulorecipes/blobstore/impl/AccumuloBlobStore.java



On Thu, Apr 10, 2014 at 7:32 AM, pdread <paul.read@siginttech.com> wrote:

> Ariel
>
> Actually we are storing anything over 128M to HDFS, as of next week. Our
> system is very large and fairly complex and I was not really intending on
> going into detail but just wondering if there was a way the Mutation thread
> to accumulo could be made more efficient.
>
> In the past we have reduced our tomcat footprint by going totally streamed
> based which increased speed and the number of clients we could handle. Most
> of our docs are in the 10-50K range but we try to process many at one time,
> plus I have 20TB of data to be processed that are over 100M per doc which
> starts to bog the system down. You have to understand we process many
> millions of docs per week and any kind of performance boost makes everyone
> happier.
>
> Thanks
>
> Paul
>
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981p8983.html
> Sent from the Users mailing list archive at Nabble.com.
>



-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*

Mime
View raw message