Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CACDjLVjBO95AXz15ZN9zOym_cEYCK_1wEZpycXHUQC+q_CV7Qg@mail.gmail.com>
References: 
 <CACDjLVjBO95AXz15ZN9zOym_cEYCK_1wEZpycXHUQC+q_CV7Qg@mail.gmail.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Thu, 21 Apr 2016 14:41:58 +0200
Message-ID: 
 <CAAdrtT3UdNrYuBZJC=28=NC98+y6Niw-ESdZX0TbG7OXEJ3CgA@mail.gmail.com>
Subject: Re: Explanation on limitations of the Flink Table API
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a11423120feaf020530fe0a2c

--001a11423120feaf020530fe0a2c
Content-Type: text/plain; charset=UTF-8

Hi Simone,

in Flink 1.0.x, the Table API does not support reading external data, i.e.,
it is not possible to read a CSV file directly from the Table API.
Tables can only be created from DataSet or DataStream which means that the
data is already converted into "Flink types".

However, the Table API is currently under heavy development as part of the
the efforts to add SQL support.
This work is taking place on the master branch and I am currently working
on interfaces to scan external data sets or ingest external data streams.
The interface will be quite generic such that it should be possible to
define a table source that reads the first lines of a file to infer
attribute names and types.
You can have a look at the current state of the API design here [1].

Feedback is welcome and can be very easily included in this phase of the
development ;-)

Cheers, Fabian

[1]
https://docs.google.com/document/d/1sITIShmJMGegzAjGqFuwiN_iw1urwykKsLiacokxSw0
<https://docs.google.com/document/d/1sITIShmJMGegzAjGqFuwiN_iw1urwykKsLiacokxSw0/edit#>

2016-04-21 14:26 GMT+02:00 Simone Robutti <simone.robutti@radicalbit.io>:

> Hello,
>
> I would like to know if it's possible to create a Flink Table from an
> arbitrary CSV (or any other form of tabular data) without doing type safe
> parsing with expliciteky type classes/POJOs.
>
> To my knowledge this is not possible but I would like to know if I'm
> missing something. My requirement is to be able to read a CSV file and
> manipulate it reading the field names from the file and inferring data
> types.
>
> Thanks,
>
> Simone
>

--001a11423120feaf020530fe0a2c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div><div><div>Hi Simone,<br><br></div><div=
>in Flink 1.0.x, the Table API does not support reading external data, i.e.=
, it is not possible to read a CSV file directly from the Table API.<br>Tab=
les can only be created from DataSet or DataStream which means that the dat=
a is already converted into &quot;Flink types&quot;.<br></div><div><br></di=
v>However, the Table API is currently under heavy development as part of th=
e the efforts to add SQL support.<br></div>This work is taking place on the=
 master branch and I am currently working on interfaces to scan external da=
ta sets or ingest external data streams.<br></div>The interface will be qui=
te generic such that it should be possible to define a table source that re=
ads the first lines of a file to infer attribute names and types.<br></div>=
You can have a look at the current state of the API design here [1].<br><br=
></div>Feedback is welcome and can be very easily included in this phase of=
 the development ;-)<br><br></div>Cheers, Fabian<br><div><div><br>[1] <a hr=
ef=3D"https://docs.google.com/document/d/1sITIShmJMGegzAjGqFuwiN_iw1urwykKs=
LiacokxSw0/edit#" target=3D"_blank">https://docs.google.com/document/d/1sIT=
IShmJMGegzAjGqFuwiN_iw1urwykKsLiacokxSw0</a><br></div></div><div class=3D"g=
mail_extra"><br><div class=3D"gmail_quote">2016-04-21 14:26 GMT+02:00 Simon=
e Robutti <span dir=3D"ltr">&lt;<a href=3D"mailto:simone.robutti@radicalbit=
.io" target=3D"_blank">simone.robutti@radicalbit.io</a>&gt;</span>:<br><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><div dir=3D"ltr">Hello,<div><br></div><div>I wou=
ld like to know if it&#39;s possible to create a Flink Table from an arbitr=
ary CSV (or any other form of tabular data) without doing type safe parsing=
 with expliciteky type classes/POJOs.=C2=A0</div><div><br></div><div>To my =
knowledge this is not possible but I would like to know if I&#39;m missing =
something. My requirement is to be able to read a CSV file and manipulate i=
t reading the field names from the file and inferring data types.=C2=A0</di=
v><div><br>Thanks,</div><div><br></div><div>Simone</div></div>
</blockquote></div><br></div></div>

--001a11423120feaf020530fe0a2c--