flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Split csv file in equally sized slices (less than or equal)
Date Mon, 28 Nov 2016 17:32:04 GMT
Hi to all,

I have a use case where I have to read a huge csv containing ids to fetch
from a table in a db.
The jdbc input format can handle parameterized queries so I was thinking to
fetch data using 1000 id at a time. What is the easiest whay to divide a
dataset by slices of 1000 ids each (in order to create parameters for my
JDBC Input format)? Is that possible?
Or maybe there's an easiest solutions using streaming APIs?

Best,
Flavio

Mime
View raw message