Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
MIME-Version: 1.0
References: <CA+-yw=+7xU0+2JkJ9K=Mnswr-HyJgs8jP=o9UZacbmR=cjH-ZQ@mail.gmail.com>
 <CAB-r6_y5xxsFJM4zBj6zP+-NEjJaFF08O+PDMEKiGUrhDGGHmg@mail.gmail.com>
In-Reply-To: <CAB-r6_y5xxsFJM4zBj6zP+-NEjJaFF08O+PDMEKiGUrhDGGHmg@mail.gmail.com>
From: Goden Yao <godenyao@apache.org>
Date: Thu, 28 Jul 2016 15:57:59 +0000
Message-ID: <CAPgpW0YPtmEYdtsN5f3_kMFkueCTWfRV9ZPgx7ge0=S7VkTkMA@mail.gmail.com>
Subject: Re: COPY command for external tables
To: dev@hawq.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a113f3d024baa8e0538b43352
archived-at: Thu, 28 Jul 2016 15:58:16 -0000

--001a113f3d024baa8e0538b43352
Content-Type: text/plain; charset=UTF-8

The scenario is
1. user creates an external writable table,
2. copy data into that external writable table.
3. query that external writable table (by creating another readable table)

I think user doesn't copy to internal tables due to some possible reasons:
the format is not supported by hawq internal tables (in JIRA , it was a
text file)
As for https://issues.apache.org/jira/browse/HAWQ-956 , I put milestone as
backlog and welcome further discussion.

On Thu, Jul 28, 2016 at 2:22 AM Lili Ma <lilima@apache.org> wrote:

> Hi Alex,
>
> Could you kindly share the user scenario for copying data to external
> table? Why don't we copy to internal table directly? Thanks :)  I thought
> external table can directly points to the file which we need copy when
> created.
>
> Thanks
> Lili
>
> 2016-07-28 6:53 GMT+08:00 Oleksandr Diachenko <odiachenko@pivotal.io>:
>
> > Hi,
> >
> > Recently we discovered that COPY command for external tables is not
> > transactional.
> > It means if user is running COPY command and during this process
> underlying
> > HDFS fails - the end state of table is inconstant.
> > As a result I filled a JIRA -
> > https://issues.apache.org/jira/browse/HAWQ-956
> > .
> >
> > From user perspective, COPY command might also need to support additional
> > options, to make import/export process more flexible.
> >
> >    - Number of error lines to tolerate.
> >    - Batch size to commit.
> >    - TBD
> >
> > Please feel free to join discussion.
> >
> > Regards, Alex.
> >
>

--001a113f3d024baa8e0538b43352--