Mailing-List: contact dev-help@sqoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@sqoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jarcec@gmail.com designates
 209.85.192.175 as permitted sender)
Sender: Jarek Jarcec Cecho <jarcec@gmail.com>
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\))
Subject: Re: Configurable NULL in IDF or Connector?
From: Jarek Jarcec Cecho <jarcec@apache.org>
In-Reply-To: 
 <CAHBV8WejsnWm6_B8Dq1OmUB-SJMJ7RNKuY5fx1S8PRTd_AG7Dg@mail.gmail.com>
Date: Mon, 1 Dec 2014 10:58:14 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <06CC5BAA-608C-41D8-8BA2-3C5F7710B849@apache.org>
References: 
 <CAOvM-cgJv7jxGOVop9yW5PP4tCcYSWuYNM7ffRbjn72gHZ3XzA@mail.gmail.com>
 <CAHBV8WejsnWm6_B8Dq1OmUB-SJMJ7RNKuY5fx1S8PRTd_AG7Dg@mail.gmail.com>
To: dev@sqoop.apache.org

I do share the same point of view as Gwen. The CSV format for UDF is =
very strict so that we have minimal surface area for inconsistencies =
between multiple connectors. This is because the IDF is an agreed upon =
exchange format when transferring data from one connector to the other. =
That however shouldn't stop one connector (such as HDFS) to offer ways =
to save the resulting CSV differently.

We had similar discussion about separator and quote characters in =
SQOOP-1522 that seems to be relevant to the NULL discussion here.

Jarcec

> On Dec 1, 2014, at 10:42 AM, Gwen Shapira <gshapira@cloudera.com> =
wrote:
>=20
> I think its two different things:
>=20
> 1. HDFS connector should give more control over the formatting of the
> data in text files (nulls, escaping, etc)
> 2. IDF should give NULLs in a format that is optimized for
> MySQL/Postgres direct connectors (since thats one of the IDF design
> goals).
>=20
> Gwen
>=20
> On Mon, Dec 1, 2014 at 9:52 AM, Abraham Elmahrek <abe@cloudera.com> =
wrote:
>> Hey guys,
>>=20
>> Any thoughts on where configurable NULL values should be? Either the =
IDF or
>> HDFS connector?
>>=20
>> cf: https://issues.apache.org/jira/browse/SQOOP-1678
>>=20
>> -Abe