flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Re: Integrate Flink with S3 on EMR cluster
Date Tue, 07 Mar 2017 17:43:32 GMT
Generally, using S3 filesystem in EMR with Flink has worked pretty well for me in Flink <
1.2 (unless you run out of connections in your HTTP pool). When you say, "using Hadoop File
System class", what do you mean? In my experience, it's sufficient to just use the "s3://"
filesystem protocol and Flink's Hadoop integration (plus S3 filesystem classes provided by
EMR) will do the right thing.

-Shannon
Mime
View raw message