Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (nike.apache.org: domain of gates@hortonworks.com
 designates 209.85.220.44 as permitted sender)
Message-ID: <54AB374B.6050904@hortonworks.com>
Date: Mon, 05 Jan 2015 17:15:55 -0800
From: Alan Gates <gates@hortonworks.com>
User-Agent: Postbox 3.0.11 (Macintosh/20140602)
MIME-Version: 1.0
To: user@hive.apache.org
Subject: Re: Hive 14 performance and scalability?
References: <BLU436-SMTP1444E760C432990FADFA76598630@phx.gbl>
In-Reply-To: <BLU436-SMTP1444E760C432990FADFA76598630@phx.gbl>
Content-Type: multipart/related;
 boundary="------------010900070502070506090300"

--------------010900070502070506090300
Content-Type: multipart/alternative;
 boundary="------------030308060808060300080409"

This is a multi-part message in MIME format.
--------------030308060808060300080409
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: quoted-printable

Mike,

I've tested it with inserts, updates, and deletes in the 100k rows per=20
partition range, and with multiple but not hundreds of users.  The thing=20
to keep in mind is the transactional changes don't affect tables that=20
you don't explicitly mark as transactional, so this shouldn't affect=20
existing installs or users who aren't interested in the functionality.

Given our design we don't expect long running transactions to be an=20
issue, other than it will prevent cleaning up old versions of the data=20
used by the long running query, which may result in temporary bloat for=20
a table or partition.  Also, keep in mind that currently all DML=20
operations are auto-commit so it is harder (though not impossible) to=20
get a long running transaction.

Alan.

> Michael Segel <mailto:msegel_hadoop@hotmail.com>
> December 11, 2014 at 3:23
> Hi,
>
> While I haven=92t upgraded to HDP 2.2, I have to ask if the transaction
> processing introduced in 14 has been tested at scale in terms of both
> users, and data size?
>
> I am curious as to what happens if you have a long transaction how well
> it copes.
>
>
> Thx
>
> -Mike

--=20
Sent with Postbox <http://www.getpostbox.com>

--=20
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to=
=20
which it is addressed and may contain information that is confidential,=20
privileged and exempt from disclosure under applicable law. If the reader=
=20
of this message is not the intended recipient, you are hereby notified that=
=20
any printing, copying, dissemination, distribution, disclosure or=20
forwarding of this communication is strictly prohibited. If you have=20
received this communication in error, please contact the sender immediately=
=20
and delete it from your system. Thank You.

--------------030308060808060300080409
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<html><head>
<meta content=3D"text/html; charset=3Dwindows-1252"=20
http-equiv=3D"Content-Type">
</head><body bgcolor=3D"#FFFFFF" text=3D"#000000">Mike,<br>
<br>
I've tested it with inserts, updates, and deletes in the 100k rows per=20
partition range, and with multiple but not hundreds of users.=A0 The thing
 to keep in mind is the transactional changes don't affect tables that=20
you don't explicitly mark as transactional, so this shouldn't affect=20
existing installs or users who aren't interested in the functionality.<br>
<br>
Given our design we don't expect long running transactions to be an=20
issue, other than it will prevent cleaning up old versions of the data=20
used by the long running query, which may result in temporary bloat for a
 table or partition.=A0 Also, keep in mind that currently all DML=20
operations are auto-commit so it is harder (though not impossible) to=20
get a long running transaction.<br>
<br>
Alan.<br>
<br>
<blockquote style=3D"border: 0px none;"=20
cite=3D"mid:BLU436-SMTP1444E760C432990FADFA76598630@phx.gbl" type=3D"cite">
  <div style=3D"margin:30px 25px 10px 25px;" class=3D"__pbConvHr"><div=20
style=3D"display:table;width:100%;border-top:1px solid=20
#EDEEF0;padding-top:5px"> 	<div=20
style=3D"display:table-cell;vertical-align:middle;padding-right:6px;"><img
 photoaddress=3D"msegel_hadoop@hotmail.com" photoname=3D"Michael Segel"=20
src=3D"cid:part1.00050108.07020601@hortonworks.com"=20
name=3D"compose-unknown-contact.jpg" height=3D"25px" width=3D"25px"></div> =
  <div
=20
style=3D"display:table-cell;white-space:nowrap;vertical-align:middle;width:=
100%">
   	<a moz-do-not-send=3D"true" href=3D"mailto:msegel_hadoop@hotmail.com"=
=20
style=3D"color:#737F92=20
!important;padding-right:6px;font-weight:bold;text-decoration:none=20
!important;">Michael Segel</a></div>   <div=20
style=3D"display:table-cell;white-space:nowrap;vertical-align:middle;">  =
=20
  <font color=3D"#9FA2A5"><span style=3D"padding-left:6px">December 11, 201=
4
 at 3:23</span></font></div></div></div>
  <div style=3D"color:#888888;margin-left:24px;margin-right:24px;"=20
__pbrmquotes=3D"true" class=3D"__pbConvBody"><div>Hi, <br><br>While I=20
haven=92t upgraded to HDP 2.2, I have to ask if the transaction <br>process=
ing
 introduced in 14 has been tested at scale in terms of both <br>users,=20
and data size? <br><br>I am curious as to what happens if you have a=20
long transaction how well  <br>it copes.<br><br><br>Thx<br><br>-Mike</div><=
/div>
</blockquote>
<br>
<div class=3D"moz-signature">-- <br>
<div>Sent with <a href=3D"http://www.getpostbox.com"><span style=3D"color:=
=20
rgb(51, 102, 153);">Postbox</span></a></div></div>
</body></html>

<br>
<span style=3D"color:rgb(128,128,128);font-family:Arial,sans-serif;font-siz=
e:10px">CONFIDENTIALITY NOTICE</span><br style=3D"color:rgb(128,128,128);fo=
nt-family:Arial,sans-serif;font-size:10px"><span style=3D"color:rgb(128,128=
,128);font-family:Arial,sans-serif;font-size:10px">NOTICE: This message is =
intended for the use of the individual or entity to which it is addressed a=
nd may contain information that is confidential, privileged and exempt from=
 disclosure under applicable law. If the reader of this message is not the =
intended recipient, you are hereby notified that any printing, copying, dis=
semination, distribution, disclosure or forwarding of this communication is=
 strictly prohibited. If you have received this communication in error, ple=
ase contact the sender immediately and delete it from your system. Thank Yo=
u.</span>
--------------030308060808060300080409--

--------------010900070502070506090300
Content-Type: image/jpeg; x-apple-mail-type=stationery;
 name="compose-unknown-contact.jpg"
Content-Transfer-Encoding: base64
Content-ID: <part1.00050108.07020601@hortonworks.com>
Content-Disposition: inline;
 filename="compose-unknown-contact.jpg"

/9j/4AAQSkZJRgABAQEARwBHAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEC
AQEBAQEBAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/2wBDAQEBAQEBAQICAgICAgIC
AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wAAR
CAAZABkDAREAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAABgcICQr/xAA0EAABAwMCAgUK
BwAAAAAAAAACAQMEBQYRABITIQcUMUF2CBUXIjI2N0JRtVRWkZOV0dL/xAAYAQEAAwEAAAAA
AAAAAAAAAAADAAEEAv/EACQRAAICAAQGAwAAAAAAAAAAAAABAhEDMrHREyExM0FxgfDx/9oA
DAMBAAIRAxEAPwDuEt+gW/ULet6oVC3rfqNQqFv0OfPn1GhUqfOmzZtKZlS5UqZMaNwzNwiJ
VIl7eXLCaZIGwBl3TY8epPx2+jy2ZNPjvkwc9uhW8j7nCPhvOsQliYIeS7cvCpp8o50qwrC4
v3lsNSDbdmTEhvs2tahxpfV3WnmbbozJEw/gwdadbYExVRXKEKoSdvJcaOSqxE7/AAiX0gXx
+a69/JSf9alIlste0VzaNpeFrcT9KKymotyiaZ0KRCnzacoE7Kjzn4gi2KqUh3jqDHDHv4mR
UfruTWlMzlVUKIVNp9GguEJnAh0+IZjyAiisgyRDnu5azS8miKqjOTVkKqS/psG37fo1Fbab
eg25b8eZPeFJBBJSjMG5HjMeyihnaauZwe4OGiju13GAcpOwBeN+U8/IkGbsiS8b7ryogmbz
hbyc9REROfZhERO5ETShjPtvpGqTUyLErytS4siSwx5x2tRH4hPOI0DkjZtaJtFxuVEbIUUi
yeNujlBUJGbJN6nM/Cyf2Hf60YgjvKA+NPSP4gT7axpcPtr51YWJnYn9dnAQWl722p4ot37y
zqnlfp6FrqbwawG8/9k=
--------------010900070502070506090300--