Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Received-SPF: pass (nike.apache.org: domain of eric.newton@gmail.com
 designates 209.85.213.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CBF50FB5.3119%ralph.perko@pnnl.gov>
References: <CBF50FB5.3119%ralph.perko@pnnl.gov>
Date: Thu, 7 Jun 2012 09:05:01 -0400
Message-ID: 
 <CADxc9Bm5ZQuowYLPbwoXDhBBVPr+-DNPgjJmvnNkNo4_OPB=pg@mail.gmail.com>
Subject: Re: Table design
From: Eric Newton <eric.newton@gmail.com>
To: user@accumulo.apache.org
Content-Type: multipart/alternative; boundary=14dae934095f12f81a04c1e1886a

--14dae934095f12f81a04c1e1886a
Content-Type: text/plain; charset=ISO-8859-1

Some thoughts:

Accumulo will accomodate keys that are very large (like 100K) but I don't
recommend it. It makes indexes big and slows down just about every
operation.  A row-id or column qualifier that is 200 bytes long is not
extreme.  Remember that compression will decrease the storage requirements,
especially since the sort creates natural redundancy in the row id.

Is it important to find "Three men and a baby" just after "Three little
pigs"?  If not, hash the title and look up the hash.  That will give you a
nice small key.  This also avoids hot-spots, like all the titles that start
with "The" or a common letter, like "S". But you may need to deal with hash
collisions.

Counters can give you "append" hot-spots.  As you ingest, the most active
tablet will always be the newest one.

A random UUID is useful, but large, if you just want a unique identifier
associated with a title.

Accumulo performance should not change if you have 1 table or 100.  But
tables are a convenient unit for management.  You can offline, compact and
delete a table.  You can configure many table-specific properties which can
give you performance benefits.

-Eric

On Wed, Jun 6, 2012 at 4:46 PM, Perko, Ralph J <Ralph.Perko@pnnl.gov> wrote:

> Hi,  I am in the process of designing some Accumulo tables for an app and
> have some questions:
>
> Lookup Table:
> The data's natural qualifier is a title.  This title can be any length.
>  Some are as long as 200 characters.
> I am using this title as a row id and also as a column qualifier in other
> places.
> Is it considered good practice to have a lookup table for these titles
> (like RDBMS), replacing them with some incremented integer value, or should
> I just continue to use these long titles as row ids?
>
> Multiple Tables:
> What are the best practices around when to create a new table?  I have
> been breaking up my tables based on row id semantics.  For example, title
> row ids are in a different table than row ids based on some analysis count.
> Does breaking up data into multiple tables, help/hurt/ or do nothing for
> accumulo performance?
>
> Thanks,
> Ralph
> __________________________________________________
> Ralph Perko
> Pacific Northwest National Laboratory
>
>
>

--14dae934095f12f81a04c1e1886a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Some thoughts:<div><br></div><div>Accumulo will accomodate keys that are ve=
ry large (like 100K) but I don&#39;t recommend it. It makes indexes big and=
 slows down just about every operation. =A0A row-id or column qualifier tha=
t is 200 bytes long is not extreme.=A0 Remember that compression will decre=
ase the storage requirements, especially since the sort creates natural red=
undancy in the row id.<br>
<div><br></div><div>Is it important to find &quot;Three men and a baby&quot=
; just after &quot;Three little pigs&quot;? =A0If not, hash the title and l=
ook up the hash. =A0That will give you a nice small key. =A0This also avoid=
s hot-spots, like all the titles that start with &quot;The&quot; or a commo=
n letter, like &quot;S&quot;. But you may need to deal with hash collisions=
.</div>
<div><br></div><div>Counters can give you &quot;append&quot; hot-spots. =A0=
As you ingest, the most active tablet will always be the newest one.</div><=
div><br></div><div>A random UUID is useful, but large, if you just want a u=
nique identifier associated with a title.</div>
<div><br></div><div>Accumulo performance should not change if you have 1 ta=
ble or 100. =A0But tables are a convenient unit for management. =A0You can =
offline, compact and delete a table. =A0You can configure many table-specif=
ic properties which can give you performance benefits.</div>
<div><br></div><div>-Eric</div><div><br><div class=3D"gmail_quote">On Wed, =
Jun 6, 2012 at 4:46 PM, Perko, Ralph J <span dir=3D"ltr">&lt;<a href=3D"mai=
lto:Ralph.Perko@pnnl.gov" target=3D"_blank">Ralph.Perko@pnnl.gov</a>&gt;</s=
pan> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi, =A0I am in the process of designing some=
 Accumulo tables for an app and have some questions:<br>
<br>
Lookup Table:<br>
The data&#39;s natural qualifier is a title. =A0This title can be any lengt=
h. =A0Some are as long as 200 characters.<br>
I am using this title as a row id and also as a column qualifier in other p=
laces.<br>
Is it considered good practice to have a lookup table for these titles (lik=
e RDBMS), replacing them with some incremented integer value, or should I j=
ust continue to use these long titles as row ids?<br>
<br>
Multiple Tables:<br>
What are the best practices around when to create a new table? =A0I have be=
en breaking up my tables based on row id semantics. =A0For example, title r=
ow ids are in a different table than row ids based on some analysis count.<=
br>

Does breaking up data into multiple tables, help/hurt/ or do nothing for ac=
cumulo performance?<br>
<br>
Thanks,<br>
Ralph<br>
__________________________________________________<br>
Ralph Perko<br>
Pacific Northwest National Laboratory<br>
<br>
<br>
</blockquote></div><br></div></div>

--14dae934095f12f81a04c1e1886a--