flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jingsong Li <jingsongl...@gmail.com>
Subject Re: changing the output files names in Streamfilesink from part-00 to something else
Date Thu, 14 May 2020 06:43:25 GMT
Hi, Dhurandar,

Can you describe your needs? Why do you need to modify file names flexibly?
What kind of name do you want?

Best,
Jingsong Lee

On Thu, May 14, 2020 at 2:05 AM dhurandar S <dhurandargeek@gmail.com> wrote:

> Yes we looked at it ,
> The problem is the file name gets generated in a dynamic fashion, based on
> which organization data we are getting we generate the file name from the
> coming data.
>
> Is there any way we can achieve this ??
>
> On Tue, May 12, 2020 at 8:38 PM Yun Gao <yungao.gy@aliyun.com> wrote:
>
>> Hi Dhurandar:
>>
>>     Currently StreamingFileSink should be able to change the prefix and
>> suffix of the filename[1], it could be changed to something like
>> <prefix>-0-0<suffix>. Could this solve your problem ?
>>
>>
>>  Best,
>>   Yun
>>
>>
>>
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/streamfile_sink.html#part-file-configuration
>>
>>
>>
>> ------------------------------------------------------------------
>> 发件人:dhurandar S<dhurandargeek@gmail.com>
>> 日 期:2020年05月13日 05:13:04
>> 收件人:user<user@flink.apache.org>; <dev@flink.apache.org>
>> 主 题:changing the output files names in Streamfilesink from part-00 to
>> something else
>>
>> We want to change the name of the file being generated as the output of
>> our StreamFileSink.
>> , when files are generated they are named part-00*, is there a way that
>> we can change the name.
>>
>> In Hadoop, we can change RecordWriters and MultipleOutputs. May I please
>> some help in this regard. This is causing blockers for us and will force us
>> t move to MR job
>>
>> --
>> Thank you and regards,
>> Dhurandar
>>
>>
>>
>
> --
> Thank you and regards,
> Dhurandar
>
>

-- 
Best, Jingsong Lee

Mime
View raw message