accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie Rinaldi <bil...@apache.org>
Subject Re: EXTERNAL: Re: Large files in Accumulo
Date Thu, 23 Aug 2012 21:33:11 GMT
The filedata example shows one way to split a file into multiple Values.

Billie


On Thu, Aug 23, 2012 at 2:05 PM, Eric Newton <eric.newton@gmail.com> wrote:

> An entire mutation needs to fit in memory several times, so you should not
> attempt to push in a single mutation larger than a 100MB unless you have a
> lot of memory in your tserver/logger.
>
> And while I'm at it, large keys will create large indexes, so try to keep
> your (row,cf,cq,cv) under 100K.
>
> -Eric
>
>
> On Thu, Aug 23, 2012 at 4:37 PM, Cardon, Tejay E <tejay.e.cardon@lmco.com>wrote:
>
>>  In my case I’ll be doing a document based index store (like the
>> wikisearch example), but my documents may be as large as several GB.  I
>> just wanted to pick the collective brain of the group to see if I’m walking
>> into a major headache.  If it’s never been tried before, then I’ll give it
>> a shot and report back.****
>>
>>
>> Tejay****
>>
>> ** **
>>
>> *From:* William Slacum [mailto:wilhelm.von.cloud@accumulo.net]
>> *Sent:* Thursday, August 23, 2012 2:07 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* EXTERNAL: Re: Large files in Accumulo****
>>
>> ** **
>>
>> Are these RFiles as a whole? I know at some point HBase needed to have
>> entire rows fit into memory; Accumulo does not have this restriction.****
>>
>> On Thu, Aug 23, 2012 at 12:55 PM, Cardon, Tejay E <
>> tejay.e.cardon@lmco.com> wrote:****
>>
>> Alright, this one’s a quick question.  I’ve been told that HBase does not
>> perform well if large (> 100MB) files are stored in it).  Does Accumulo
>> have similar trouble?  If so, can it be overcome by storing the large files
>> in their own locality group?****
>>
>>  ****
>>
>> Thanks,****
>>
>> Tejay****
>>
>> ** **
>>
>
>

Mime
View raw message