beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1281) GlobalWindow needs non-empty encoding in StateNamespaces
Date Thu, 19 Jan 2017 03:24:26 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829235#comment-15829235
] 

ASF GitHub Bot commented on BEAM-1281:
--------------------------------------

GitHub user kennknowles opened a pull request:

    https://github.com/apache/beam/pull/1793

    [BEAM-1281] Encode GlobalWindow in one byte when it is the whole stream

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [x] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    See JIRA and inline comment for explanation. TL;DR empty encodings are pretty fragile.
Many coders are sensitive to one byte addition, but this will not affect anything in the main
element collections, as they are within a `WindowedValueCoder`. The motivating case is actually
quite mundane.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kennknowles/beam GlobalWindow-encoding

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/1793.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1793
    
----
commit 219d0cc8f5094240d2eafab6d2c1579aa2c70219
Author: Kenneth Knowles <klk@google.com>
Date:   2017-01-19T03:20:36Z

    Encoding GlobalWindow in one byte when it is the whole stream

----


> GlobalWindow needs non-empty encoding in StateNamespaces
> --------------------------------------------------------
>
>                 Key: BEAM-1281
>                 URL: https://issues.apache.org/jira/browse/BEAM-1281
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>            Priority: Minor
>
> Because the GlobalWindow is encoded to zero bytes, a StateNamespace built from the window
has a stringKey "//" while the global namespace's stringKey is "/". As paths, these are identical,
though we don't currently treat them as paths, quite. It isn't clear whether this is desirable.
Maybe it is harmless but it complicates parsing and interpretation.
> For a system that actually builds hierarchical paths out of, say, some prefix, the StateNamespace,
and a subsequent ID, the canonicalized path is the same so it is not possible to deserialize
to the original.
> There are other gotchas associated with zero-length encodings, such as APIs that return
zero bytes when there is no more data ready, versus returning zero bytes because the data
is representable in zero bytes, given the context of knowing what type of data is expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message