beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <>
Subject [jira] [Created] (BEAM-3202) Multiple deserializations of PipelineOptions leaks memory
Date Thu, 16 Nov 2017 17:24:00 GMT
Luke Cwik created BEAM-3202:

             Summary: Multiple deserializations of PipelineOptions leaks memory
                 Key: BEAM-3202
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core
            Reporter: Luke Cwik
             Fix For: 2.3.0

In particular, upon deserializing a PipelineOptions object,
calls ValueProvider.RuntimeValueProvider.setRuntimeOptions(options) which
stores the (newly) deserialized PipelineOptions instance in a static map
inside the RuntimeValueProvider class, where the key is an id obtained by
calling deserializedOptions.getOptionsId().

The thing is, performing a serialize-deserialize cycle on a given
PipelineOptions instance and invoking getOptionsId() yields different
optionsIds. Therefore, multiple deserializations of the same
PipelineOptions instance result in new keys being added to the static
"optionsMap" map inside the ValueProvider.RuntimeValueProvider class.

The fix is to populate the options id when PipelineOptions is created. This can be tested
by creating a PipelineOptions object and then serializing/deserializing it and ensuring that
it has the same options id as the original.

This message was sent by Atlassian JIRA

View raw message