flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Congxian Qiu(klion26) (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-18038) StateBackendLoader logs application-defined state before it is fully configured
Date Mon, 01 Jun 2020 05:52:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120765#comment-17120765

Congxian Qiu(klion26) commented on FLINK-18038:

I think logging the final used configuration is useful for the user.

> StateBackendLoader logs application-defined state before it is fully configured
> -------------------------------------------------------------------------------
>                 Key: FLINK-18038
>                 URL: https://issues.apache.org/jira/browse/FLINK-18038
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / State Backends
>    Affects Versions: 1.9.1
>            Reporter: Steve Bairos
>            Priority: Trivial
> In the [StateBackendLoader|[https://github.com/apache/flink/blob/bb46756b84940a6134910e74406bfaff4f2f37e9/flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java#L201]],
there's this log line:
> {code:java}
> logger.info("Using application-defined state backend: {}", fromApplication); {code}
> It seems like this is inaccurate though because immediately after logging this, if fromApplication
is a ConfigurableStateBackend, we call the .configure() function and it is replaced by a newly
configured StateBackend. 
> To me, it seems like it would be better if we logged the state backend after it was fully
configured. In the current setup, we get confusing logs like this: 
> {code:java}
> 2020-05-29 21:39:44,387 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask   
       - Using application-defined state backend: RocksDBStateBackend{checkpointStreamBackend=File
State Backend (checkpoints: 's3://pinterest-montreal/checkpoints/xenon-dev-001-20191210/Xenon/BasicJavaStream',
savepoints: 'null', asynchronous: UNDEFINED, fileStateThreshold: -1), localRocksDbDirectories=null,
enableIncrementalCheckpointing=UNDEFINED, numberOfTransferingThreads=-1}2020-05-29 21:39:44,387
INFO  org.apache.flink.streaming.runtime.tasks.StreamTask           - Configuring application-defined
state backend with job/cluster config{code}
> Which makes it ambiguous whether or not settings in our flink-conf.yaml like "state.backend.incremental:
true" are being applied properly or not. 
> I can make a diff for the change if there aren't any objections

This message was sent by Atlassian Jira

View raw message