flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Villalobos <mvillalo...@kineteque.com>
Subject Re: Does Flink support reading files or CSV files from java.io.InputStream instead of file paths?
Date Wed, 17 Jun 2020 14:15:25 GMT
While I still think it would be great for Flink to accept an InputStream, and allow the programmer
to decide if it is a remote TCP call or local file, for the sake of my demo, I simply 
found the file path within Gradle and supplied to the Gradle application run plugin like this:

run {
    args = ["--input-file", file('timeseries.csv')]
}

and that launched my application with minimal configuration.

> On Jun 17, 2020, at 7:11 AM, Aljoscha Krettek <aljoscha@apache.org> wrote:
> 
> Hi,
> 
> for simple demos you can also use env.fromElements() or env.fromCollection() to create
a source from some data that you have already available.
> 
> Does that help?
> 
> Best,
> Aljoscha
> 
> On 16.06.20 15:35, Marco Villalobos wrote:
>> Okay, it is not supported.
>> I understand such a feature is not needed in production systems, but it could make
testing and demos more portable. I was writing a demo, and I wanted it to run without command-line
arguments, which would have been very handy. I want the user to simply checkout the code and
run it without having to supply a command line parameter declaring where the input file resides.
>> Thank you.
>>> On Jun 16, 2020, at 4:57 AM, Aljoscha Krettek <aljoscha@apache.org> wrote:
>>> 
>>> Hi Marco,
>>> 
>>> this is not possible since Flink is designed mostly to read files from a distributed
filesystem, where paths are used to refer to those files. If you read from files on the classpath
you could just use plain old Java code and won't need a distributed processing system such
as Flink.
>>> 
>>> Best,
>>> Aljoscha
>>> 
>>> On 16.06.20 06:46, Marco Villalobos wrote:
>>>> Does Flink support reading files or CSV files from java.io.InputStream instead
of file paths?
>>>> I'd rather just store my file on the class path and load it with java.lang.ClassLoader#getResourceAsStream(String).
>>>> If there is a way, I'd appreciate an example.
>>> 
> 


Mime
View raw message