Return-Path: 
 <user-return-11435-apmail-hive-user-archive=hive.apache.org@hive.apache.org>
X-Original-To: apmail-hive-user-archive@www.apache.org
Delivered-To: apmail-hive-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 925F0E379
	for <apmail-hive-user-archive@www.apache.org>;
 Mon, 24 Dec 2012 13:09:54 +0000 (UTC)
Received: (qmail 26761 invoked by uid 500); 24 Dec 2012 13:09:52 -0000
Delivered-To: apmail-hive-user-archive@hive.apache.org
Received: (qmail 26444 invoked by uid 500); 24 Dec 2012 13:09:48 -0000
Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hive.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hive.apache.org>
List-Post: <mailto:user@hive.apache.org>
List-Id: <user.hive.apache.org>
Reply-To: user@hive.apache.org
Delivered-To: mailing list user@hive.apache.org
Received: (qmail 26417 invoked by uid 99); 24 Dec 2012 13:09:48 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 24 Dec 2012 13:09:48 +0000
X-ASF-Spam-Status: No, hits=2.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_SOFTFAIL
X-Spam-Check-By: apache.org
Received-SPF: softfail (nike.apache.org: transitioning domain of
 iyakti@souq.com does not designate 209.85.215.53 as permitted sender)
Received: from [209.85.215.53] (HELO mail-la0-f53.google.com) (209.85.215.53)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 24 Dec 2012 13:09:40 +0000
Received: by mail-la0-f53.google.com with SMTP id w12so8361878lag.26
        for <user@hive.apache.org>; Mon, 24 Dec 2012 05:09:19 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=google.com; s=20120113;
        h=mime-version:from:date:message-id:subject:to:content-type
         :x-gm-message-state;
        bh=SYnXXCnQ4+DoYX+5+fiImlcFpVd87M93nC0ix/xXQT0=;
        b=iwcnjVeZqU97Pcg8fwFvlsEeS2kR7wbpEaWpVxtyi4cx+cwvXSZzwFuBBykr8XXHJX
         28a1RgRZWughPTveR70QjDqXanqejaVA2Nzvb4DzOWBZmt2H9PNm9GDO50GE4gfcBHon
         zm1ZAzQebfO8YgvQD/QC53/09pwzBblhcWIoN083J1YSOzvQxcocbBpEIexKTPsde8GF
         SFzI73SSFcYJIvIkEtDwp8mVbtdE90Glcit/jvsP8ftTZg2LxYrz/S9p6XHPnJKMq/A5
         BZxGJvI/KqsgfzxacdXOImkH8dWcQ53JB9rkudGvwCOtaIcwaDsKAl2w6/CjoHhHi+Mq
         9COA==
Received: by 10.112.29.229 with SMTP id n5mr8882802lbh.130.1356354559572; Mon,
 24 Dec 2012 05:09:19 -0800 (PST)
MIME-Version: 1.0
Received: by 10.114.13.201 with HTTP; Mon, 24 Dec 2012 05:08:39 -0800 (PST)
From: Ibrahim Yakti <iyakti@souq.com>
Date: Mon, 24 Dec 2012 16:08:39 +0300
Message-ID: 
 <CAG9O-W1ixp1F-YuRK0xk3OtjBM03JmkO2AUm3spMgkwCWiVP9A@mail.gmail.com>
Subject: Reflect MySQL updates into Hive
To: user@hive.apache.org
Content-Type: multipart/alternative; boundary=f46d0421a78bb67a7904d198e789
X-Gm-Message-State: 
 ALoCoQnMsI3bo0W2e20+Jco6ZE36NGT0+w5pNNc66/PzMDG86TPNXdVXotyvWrufDNkXzCawK0gP
X-Virus-Checked: Checked by ClamAV on apache.org

--f46d0421a78bb67a7904d198e789
Content-Type: text/plain; charset=UTF-8

Hi All,

We are new to hadoop and hive, we are trying to use hive to
run analytical queries and we are using sqoop to import data into hive, in
our RDBMS the data updated very frequently and this needs to be reflected
to hive. Hive does not support update/delete but there are many workarounds
to do this task.

What's in our mind is importing all the tables into hive as is, then we
build the required tables for reporting.

My questions are:

   1. What is the best way to reflect MySQL updates into Hive with minimal
   resources?
   2. Is sqoop the right tool to do the ETL?
   3. Is Hive the right tool to do this kind of queries or we should search
   for alternatives?

Any hint will be useful, thanks in advanced.

--
Ibrahim

--f46d0421a78bb67a7904d198e789
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi All,<div><br></div><div>We are new to hadoop and hive, =
we are trying to use hive to run=C2=A0analytical=C2=A0queries and we are us=
ing sqoop to import data into hive, in our RDBMS the data updated very freq=
uently and this needs to be reflected to hive. Hive does not support update=
/delete but there are many workarounds to do this task.</div>

<div><br></div><div style>What&#39;s in our mind is importing all the table=
s into hive as is, then we build the required tables for reporting.</div><d=
iv><br></div><div>My questions are:=C2=A0</div><div><ol style><li style>Wha=
t is the best way to reflect MySQL updates into Hive with minimal resources=
?</li>

<li style>Is sqoop the right tool to do the ETL?<br></li><li style>Is Hive =
the right tool to do this kind of queries or we should search for alternati=
ves?</li></ol><div><div style>Any hint will be useful, thanks in advanced.<=
/div>

<div dir=3D"ltr"><br><div><font color=3D"#666666">--</font></div><div><font=
 color=3D"#666666">Ibrahim</font></div></div></div>
</div></div>

--f46d0421a78bb67a7904d198e789--