Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B0B5F10E81 for ; Mon, 28 Oct 2013 18:12:02 +0000 (UTC) Received: (qmail 51650 invoked by uid 500); 28 Oct 2013 18:12:01 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 51597 invoked by uid 500); 28 Oct 2013 18:12:00 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 51589 invoked by uid 99); 28 Oct 2013 18:11:59 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 18:11:58 +0000 Received: from localhost (HELO mail-la0-f45.google.com) (127.0.0.1) (smtp-auth username vines, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 18:11:58 +0000 Received: by mail-la0-f45.google.com with SMTP id hp15so5544219lab.18 for ; Mon, 28 Oct 2013 11:11:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=cumLTHSKkPVMAFHCfKphms3dpejxUBsSp53fMiOC3Pc=; b=bT+ZDQJi+55AkQBwQoAS6ljYjqHmsWQw4MY3Jb8tgaR1TLvV8ZrZrFWqSAmE6KWBd6 13YGvy6W5hpXZv7VMDAvNWOAMXqtbkXRUzxDgF4NptRh8KWxdK1ZAVgsFlXnYNNMB6Sr zLIUxdpAbWQcbH6nFaZMZdHT/pJA8yIazL1X6hPnbemDLkT1zmKe79NYLXGt2uv8dJ7C amXbV8so1/GnSHat24h49/QmuLLBj2hQyQ47fCaSTG/eczVs96CMJ5tsytruWkec0Cvm mHMLzRqyGd3AbTPT4euVYuUP719D6jEOR3N2ziBme5YbTjmDpwKd7AyUqRkBmuJ2eCtl pSGw== X-Received: by 10.112.198.199 with SMTP id je7mr20892lbc.53.1382983915586; Mon, 28 Oct 2013 11:11:55 -0700 (PDT) MIME-Version: 1.0 Reply-To: vines@apache.org Received: by 10.114.99.99 with HTTP; Mon, 28 Oct 2013 11:11:15 -0700 (PDT) In-Reply-To: References: From: John Vines Date: Mon, 28 Oct 2013 14:11:15 -0400 Message-ID: Subject: Re: Max tablet size & Pre-splitting To: "user@accumulo.apache.org" Content-Type: multipart/alternative; boundary=001a11c2ae1804ccbd04e9d10923 --001a11c2ae1804ccbd04e9d10923 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable There is no hardcoded maximum for file size in Accumulo, so the split threshhold is the only things that provides some sort of definition for tablet size. Please be aware, if you have giant rows, you can have a tablet that exceeds the split threshhold as well, hence me referring to it loosely as the defining characteristic. As for tablet size, you can get that information from the !METADATA table, as one option. Eric Newton recently wrote a reply on this mailing list in the past 2 weeks, I think, which explained the entries there. On Mon, Oct 28, 2013 at 1:59 PM, Slater, David M. wrote: > First, a quick question: For Accumulo 1.4.2, is there a maximum size that > tablet can have? In other words, if I was to do something like > table.split.threshold=3D1000G, would that actually allow the tablet to gr= ow > to that size, or is there some static maximum, like 2G that a tablet can > have?**** > > ** ** > > The reason I ask this is that I=92m doing time-based presplitting of tabl= es, > so that I add a set of split points when I get to a new time range (or on= e > of the tablets reach a certain size), and then transfer all of my ingest = to > the new set of tablets created. This keeps me from needing to do any tabl= e > splits involving data. Therefore, I would like to set the table split > threshold arbitrarily high, so that my presplitting algorithm can do all > the work.**** > > ** ** > > Second, is there a preferred way to estimate the tablet sizes from the > Java API? I have the Ingestion application using my split points and > mutation.numBytes() to keep track of the number of bytes per tablet. Shou= ld > I be using mutation.memory() instead? Or is there a more direct way via > connector.tableOperations() or some other mechanism to determine the size > of the tablet?**** > > ** ** > > Thanks,**** > > David**** > --001a11c2ae1804ccbd04e9d10923 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
There is no hardcoded maximum for file size in Accumulo, s= o the split threshhold is the only things that provides some sort of defini= tion for tablet size. Please be aware, if you have giant rows, you can have= a tablet that exceeds the split threshhold as well, hence me referring to = it loosely as the defining characteristic.


As for tablet size, you can get that informat= ion from the !METADATA table, as one option. Eric Newton recently wrote a r= eply on this mailing list in the past 2 weeks, I think, which explained the= entries there.


On Mon,= Oct 28, 2013 at 1:59 PM, Slater, David M. <David.Slater@jhuapl.edu<= /a>> wrote:

First, a= quick question: For Accumulo 1.4.2, is there a maximum size that tablet ca= n have? In other words, if I was to do something like table.split.threshold= =3D1000G, would that actually allow the tablet to grow to that size, or is = there some static maximum, like 2G that a tablet can have?

=A0<= /p>

The reason I ask this = is that I=92m doing time-based presplitting of tables, so that I add a set = of split points when I get to a new time range (or one of the tablets reach= a certain size), and then transfer all of my ingest to the new set of tabl= ets created. This keeps me from needing to do any table splits involving da= ta. Therefore, I would like to set the table split threshold arbitrarily hi= gh, so that my presplitting algorithm can do all the work.

=A0<= /p>

Second, is there a pre= ferred way to estimate the tablet sizes from the Java API? I have the Inges= tion application using my split points and mutation.numBytes() to keep trac= k of the number of bytes per tablet. Should I be using mutation.memory() in= stead? Or is there a more direct way via connector.tableOperations() or som= e other mechanism to determine the size of the tablet?=

=A0<= /p>

Thanks,<= /span>

David


--001a11c2ae1804ccbd04e9d10923--