accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: EXTERNAL: Re: Large files in Accumulo
Date Thu, 23 Aug 2012 21:05:48 GMT
An entire mutation needs to fit in memory several times, so you should not
attempt to push in a single mutation larger than a 100MB unless you have a
lot of memory in your tserver/logger.

And while I'm at it, large keys will create large indexes, so try to keep
your (row,cf,cq,cv) under 100K.

-Eric

On Thu, Aug 23, 2012 at 4:37 PM, Cardon, Tejay E <tejay.e.cardon@lmco.com>wrote:

>  In my case I’ll be doing a document based index store (like the
> wikisearch example), but my documents may be as large as several GB.  I
> just wanted to pick the collective brain of the group to see if I’m walking
> into a major headache.  If it’s never been tried before, then I’ll give it
> a shot and report back.****
>
>
> Tejay****
>
> ** **
>
> *From:* William Slacum [mailto:wilhelm.von.cloud@accumulo.net]
> *Sent:* Thursday, August 23, 2012 2:07 PM
> *To:* user@accumulo.apache.org
> *Subject:* EXTERNAL: Re: Large files in Accumulo****
>
> ** **
>
> Are these RFiles as a whole? I know at some point HBase needed to have
> entire rows fit into memory; Accumulo does not have this restriction.****
>
> On Thu, Aug 23, 2012 at 12:55 PM, Cardon, Tejay E <tejay.e.cardon@lmco.com>
> wrote:****
>
> Alright, this one’s a quick question.  I’ve been told that HBase does not
> perform well if large (> 100MB) files are stored in it).  Does Accumulo
> have similar trouble?  If so, can it be overcome by storing the large files
> in their own locality group?****
>
>  ****
>
> Thanks,****
>
> Tejay****
>
> ** **
>

Mime
View raw message