flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Sqoop-like module in Flink
Date Wed, 13 Apr 2016 10:45:46 GMT
Hi to all,
we've recently migrated our sqoop[1] import process to a Flink job, using
an improved version of the Flink JDBC Input Format[2] that is able to
exploit the parallelism of the cluster (the current Flink version
implements NonParallelInput).

Still need to improve the mapping part of sql types to java ones (in the
addValue method IMHO) but this could be the basis for a flink-sqoop module
that will incrementally cover the sqoop functionalities when requested.
Do you think that such a module could be of interest for Flink or not?

[1] https://sqoop.apache.org/
[2] https://gist.github.com/fpompermaier/bcd704abc93b25b6744ac76ac17ed351

Best,
Flavio

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message