Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
MIME-Version: 1.0
In-Reply-To: <2025950601.3106319.1460154291292.JavaMail.yahoo@mail.yahoo.com>
References: <2025950601.3106319.1460154291292.JavaMail.yahoo.ref@mail.yahoo.com>
 <2025950601.3106319.1460154291292.JavaMail.yahoo@mail.yahoo.com>
From: Elliot West <teabot@gmail.com>
Date: Thu, 26 May 2016 07:28:29 +0100
Message-ID: <CAC3gpCYRZfMDazLg=-8qOHJmx5Lna-mQ3BBaCkcNHNCBVgc9AA@mail.gmail.com>
Subject: Re: Copying all Hive tables from Prod to UAT
To: "user@hive.apache.org" <user@hive.apache.org>, Ashok Kumar <ashok34668@yahoo.com>
Content-Type: multipart/alternative; boundary=94eb2c000bec303a050533b8e730
archived-at: Thu, 26 May 2016 06:28:56 -0000

--94eb2c000bec303a050533b8e730
Content-Type: text/plain; charset=UTF-8

Hello,

I've been looking at this recently for moving Hive tables from on-premise
clusters to the cloud, but the principle should be the same for your
use-case. If you wish to do this in an automated way, some tools worth
considering are:

   - Hive's built in replication framework:
   https://cwiki.apache.org/confluence/display/Hive/Replication
   - Hive's IMPORT/EXPORT primitives:
   https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport
   - AirBnB's ReAir replication tool:
   https://medium.com/airbnb-engineering/reair-easy-to-use-tools-for-migrating-and-replicating-petabyte-scale-data-warehouses-5153f8a433da

Elliot.

On 8 April 2016 at 23:24, Ashok Kumar <ashok34668@yahoo.com> wrote:

> Hi,
>
> Anyone has suggestions how to create and copy Hive and Spark tables from
> Production to UAT.
>
> One way would be to copy table data to external files and then move the
> external files to a local target directory and populate the tables in
> target Hive with data.
>
> Is there an easier way of doing so?
>
> thanks
>
>
>

--94eb2c000bec303a050533b8e730
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello,<div><br></div><div>I&#39;ve been looking at this re=
cently for moving Hive tables from on-premise clusters to the cloud, but th=
e principle should be the same for your use-case. If you wish to do this in=
 an automated way, some tools worth considering are:</div><div><ul><li>Hive=
&#39;s built in replication framework:=C2=A0<a href=3D"https://cwiki.apache=
.org/confluence/display/Hive/Replication">https://cwiki.apache.org/confluen=
ce/display/Hive/Replication</a></li><li>Hive&#39;s IMPORT/EXPORT primitives=
:=C2=A0<a href=3D"https://cwiki.apache.org/confluence/display/Hive/Language=
Manual+ImportExport">https://cwiki.apache.org/confluence/display/Hive/Langu=
ageManual+ImportExport</a></li><li>AirBnB&#39;s ReAir replication tool:=C2=
=A0<a href=3D"https://medium.com/airbnb-engineering/reair-easy-to-use-tools=
-for-migrating-and-replicating-petabyte-scale-data-warehouses-5153f8a433da"=
>https://medium.com/airbnb-engineering/reair-easy-to-use-tools-for-migratin=
g-and-replicating-petabyte-scale-data-warehouses-5153f8a433da</a></li></ul>=
<div>Elliot.</div></div></div><div class=3D"gmail_extra"><br><div class=3D"=
gmail_quote">On 8 April 2016 at 23:24, Ashok Kumar <span dir=3D"ltr">&lt;<a=
 href=3D"mailto:ashok34668@yahoo.com" target=3D"_blank">ashok34668@yahoo.co=
m</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style=
=3D"color:#000;background-color:#fff;font-family:times new roman,new york,t=
imes,serif;font-size:16px"><div>Hi,</div><div><br></div><div dir=3D"ltr">An=
yone has suggestions how to create and copy Hive and Spark tables from Prod=
uction to UAT.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">One way wou=
ld be to copy table data to external files and then move the external files=
 to a local target directory and populate the tables in target Hive with da=
ta.</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Is there an easier way=
 of doing so?</div><div dir=3D"ltr"><br></div><div dir=3D"ltr">thanks</div>=
<div dir=3D"ltr"><br></div><div dir=3D"ltr"><br></div></div></div></blockqu=
ote></div><br></div>

--94eb2c000bec303a050533b8e730--