flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-7643) Configure FileSystems only once
Date Wed, 04 Oct 2017 10:53:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stephan Ewen updated FLINK-7643:
--------------------------------
    Description: 
HadoopFileSystem always reloads GlobalConfiguration, which potentially leads to a lot of noise
in the logs, because this happens on each checkpoint.

Instead, file systems should be configured once upon process startup, when the configuration
is loaded.

This will also increase efficiency of checkpoints, as it avoids redundant parsing for each
data chunk.

  was:
HadoopFileSystem always reloads GlobalConfiguration, which potentially leads to a lot of noise
in the logs, because this happens on each checkpoint.

Instead, file systems should be configured once upon process startup, when the configuration
is loaded.


> Configure FileSystems only once
> -------------------------------
>
>                 Key: FLINK-7643
>                 URL: https://issues.apache.org/jira/browse/FLINK-7643
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.4.0
>            Reporter: Ufuk Celebi
>            Assignee: Stephan Ewen
>
> HadoopFileSystem always reloads GlobalConfiguration, which potentially leads to a lot
of noise in the logs, because this happens on each checkpoint.
> Instead, file systems should be configured once upon process startup, when the configuration
is loaded.
> This will also increase efficiency of checkpoints, as it avoids redundant parsing for
each data chunk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message