accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pdread <paul.r...@siginttech.com>
Subject Re: Stream fed accumulo
Date Thu, 10 Apr 2014 11:32:36 GMT
Ariel 

Actually we are storing anything over 128M to HDFS, as of next week. Our
system is very large and fairly complex and I was not really intending on
going into detail but just wondering if there was a way the Mutation thread
to accumulo could be made more efficient.

In the past we have reduced our tomcat footprint by going totally streamed
based which increased speed and the number of clients we could handle. Most
of our docs are in the 10-50K range but we try to process many at one time,
plus I have 20TB of data to be processed that are over 100M per doc which
starts to bog the system down. You have to understand we process many
millions of docs per week and any kind of performance boost makes everyone
happier. 

Thanks

Paul




--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Stream-fed-accumulo-tp8981p8983.html
Sent from the Users mailing list archive at Nabble.com.

Mime
View raw message