Return-Path: 
 <user-return-3258-apmail-accumulo-user-archive=accumulo.apache.org@accumulo.apache.org>
X-Original-To: apmail-accumulo-user-archive@www.apache.org
Delivered-To: apmail-accumulo-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 03A5D10600
	for <apmail-accumulo-user-archive@www.apache.org>;
 Tue, 29 Oct 2013 21:50:43 +0000 (UTC)
Received: (qmail 20712 invoked by uid 500); 29 Oct 2013 21:50:42 -0000
Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org
Received: (qmail 20681 invoked by uid 500); 29 Oct 2013 21:50:42 -0000
Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@accumulo.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@accumulo.apache.org>
List-Post: <mailto:user@accumulo.apache.org>
List-Id: <user.accumulo.apache.org>
Reply-To: user@accumulo.apache.org
Delivered-To: mailing list user@accumulo.apache.org
Received: (qmail 20673 invoked by uid 99); 29 Oct 2013 21:50:42 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 21:50:42 +0000
X-ASF-Spam-Status: No, hits=-0.1 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of David.Slater@jhuapl.edu
 designates 128.244.251.36 as permitted sender)
Received: from [128.244.251.36] (HELO pilot.jhuapl.edu) (128.244.251.36)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 21:50:33 +0000
Received: from aplexcas1.dom1.jhuapl.edu (unknown [128.244.198.90]) by
 pilot.jhuapl.edu with smtp
	(TLS: TLSv1/SSLv3,128bits,RC4-MD5)
	 id 75a5_ec9b_440a3c1d_22de_4a35_94e4_2fdef56b7963;
	Tue, 29 Oct 2013 17:50:09 -0400
Received: from aplesstripe.dom1.jhuapl.edu ([128.244.198.211]) by
 aplexcas1.dom1.jhuapl.edu ([128.244.198.90]) with mapi; Tue, 29 Oct 2013
 17:50:09 -0400
From: "Slater, David M." <David.Slater@jhuapl.edu>
To: "user@accumulo.apache.org" <user@accumulo.apache.org>
Date: Tue, 29 Oct 2013 17:50:08 -0400
Subject: sum of mutation.numBytes() significantly different from rfile size
Thread-Topic: sum of mutation.numBytes() significantly different from rfile
 size
Thread-Index: Ac7U6xqtdAAjLfzwQIKYvX0izPoMBA==
Message-ID: 
 <AC78983C72177B4D9D1C14F7F4AEBA2144915A15D3@aplesstripe.dom1.jhuapl.edu>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_AC78983C72177B4D9D1C14F7F4AEBA2144915A15D3aplesstripedo_"
MIME-Version: 1.0
X-Virus-Checked: Checked by ClamAV on apache.org

--_000_AC78983C72177B4D9D1C14F7F4AEBA2144915A15D3aplesstripedo_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hello,

I'm seeing about an order of magnitude difference between the number of byt=
es returned by mutation.numBytes() and the size of the rfiles on disk (Accu=
mulo 1.4.2). Note that all of my mutations are new entries, and there are n=
o combiners running.

While I understand that there is some compression on the rfile, I would be =
really surprised if it was 10:1.

My entries are composed of a row ID (most of which is equivalent to the pre=
vious row ID), an empty column family, a nonempty column qualifier (which l=
ikely shares a lot with the previous qualifier), and an empty value. An exa=
mple of the rowID and column qualifier might be:

(forward table)
0000000000000|9|fa19                 IP|127.000.000.001
0000000000000|9|fa19                  PORT|00080
...
0000000000000|9|fa22                  IP|128.032.144.139
...
<timeblock>|<hash>|<uid>       <index>|<textual value>

OR
(reverse table)
0000000000000|IP|127.000.000.001         fa19
0000000000000|IP|127.000.000.001         fd02
0000000000000|IP|127.000.000.002         123
...
0000000000000|PORT|00080                      fa19

The numBytes() method appears to return a number of bytes equal to the stri=
ng length of the row ID and column qualifiers, plus 26 * # of column qualif=
iers.

Is there something else that I'm missing, or would this possibly compress b=
y that much?

Thanks,
David

--_000_AC78983C72177B4D9D1C14F7F4AEBA2144915A15D3aplesstripedo_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content=
=3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros=
oft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-US link=3Dblue vli=
nk=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal><span style=3D'f=
ont-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hello,<o:=
p></o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;fon=
t-family:"Calibri","sans-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p>=
<p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri",=
"sans-serif";color:#1F497D'>I&#8217;m seeing about an order of magnitude di=
fference between the number of bytes returned by mutation.numBytes() and th=
e size of the rfiles on disk (Accumulo 1.4.2). Note that all of my mutation=
s are new entries, and there are no combiners running. <o:p></o:p></span></=
p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri=
","sans-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNor=
mal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";colo=
r:#1F497D'>While I understand that there is some compression on the rfile, =
I would be really surprised if it was 10:1. <o:p></o:p></span></p><p class=
=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-se=
rif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span =
style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'=
>My entries are composed of a row ID (most of which is equivalent to the pr=
evious row ID), an empty column family, a nonempty column qualifier (which =
likely shares a lot with the previous qualifier), and an empty value. An ex=
ample of the rowID and column qualifier might be:<o:p></o:p></span></p><p c=
lass=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","san=
s-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><s=
pan style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F4=
97D'>(forward table)<o:p></o:p></span></p><p class=3DMsoNormal><span style=
=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>0000=
000000000|9|fa19  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; IP|127.000.000.001<o:p></o:p></span></p><=
p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","=
sans-serif";color:#1F497D'>0000000000000|9|fa19 &nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; PORT|=
00080<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11=
.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>&#8230;<o:p></o:p></=
span></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"=
Calibri","sans-serif";color:#1F497D'>0000000000000|9|fa22 &nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp; IP|128.032.144.139<o:p></o:p></span></p><p class=3DMsoNormal><span sty=
le=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>&#=
8230;<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11=
.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>&lt;timeblock&gt;|&l=
t;hash&gt;|&lt;uid&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;index&gt;|&lt;tex=
tual value&gt;<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'fon=
t-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p>&nbsp;=
</o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-=
family:"Calibri","sans-serif";color:#1F497D'>OR<o:p></o:p></span></p><p cla=
ss=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-=
serif";color:#1F497D'>(reverse table)<o:p></o:p></span></p><p class=3DMsoNo=
rmal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";col=
or:#1F497D'>0000000000000|IP|127.000.000.001 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp; fa19<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'=
font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>00000000=
00000|IP|127.000.000.001&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; fd=
02<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11.0p=
t;font-family:"Calibri","sans-serif";color:#1F497D'>0000000000000|IP|127.00=
0.000.002&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 123<o:p></o:p></s=
pan></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"C=
alibri","sans-serif";color:#1F497D'>&#8230;<o:p></o:p></span></p><p class=
=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-se=
rif";color:#1F497D'>0000000000000|PORT|00080&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp; fa19<o:p></o:p></span></p><p class=3DMsoNormal><span s=
tyle=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>=
<o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:1=
1.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The numBytes() meth=
od appears to return a number of bytes equal to the string length of the ro=
w ID and column qualifiers, plus 26 * # of column qualifiers. <o:p></o:p></=
span></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"=
Calibri","sans-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=
=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-se=
rif";color:#1F497D'>Is there something else that I&#8217;m missing, or woul=
d this possibly compress by that much?<o:p></o:p></span></p><p class=3DMsoN=
ormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";co=
lor:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span style=
=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Than=
ks,<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D'font-size:11.0=
pt;font-family:"Calibri","sans-serif";color:#1F497D'>David<o:p></o:p></span=
></p></div></body></html>=

--_000_AC78983C72177B4D9D1C14F7F4AEBA2144915A15D3aplesstripedo_--