apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ilganeli <...@git.apache.org>
Subject [GitHub] apex-malhar pull request #316: High-level WindowedStream API (Review only)
Date Thu, 09 Jun 2016 21:06:47 GMT
Github user ilganeli commented on a diff in the pull request:

    https://github.com/apache/apex-malhar/pull/316#discussion_r66521781
  
    --- Diff: library/src/main/java/com/datatorrent/lib/window/WindowOption.java ---
    @@ -0,0 +1,143 @@
    +package com.datatorrent.lib.window;
    +
    +import java.util.LinkedList;
    +import java.util.List;
    +
    +/**
    + *
    + */
    +public abstract class WindowOption
    +{
    +
    +  private TriggerOption[] triggerOptions;
    +
    +  private AccumulationMode accumulationMode = AccumulationMode.ACCUMULATE;
    +
    +  private List<Object[]> lateness = new LinkedList<>();
    +
    +  public static enum AccumulationMode
    +  {
    +    DISCARD,
    +    ACCUMULATE,
    +    ACCUMULATE_DELTA
    +  }
    +
    +  public static class GlobalWindow extends WindowOption
    +  {
    +
    +  }
    +
    +  public static class FixedWindow extends WindowOption
    +  {
    +    FixedWindow(long quantity, Quantification.Unit unit)
    +    {
    +      size.add(new Object[]{quantity, unit});
    +    }
    +
    +    FixedWindow(List<Object[]> size)
    +    {
    +      this.size = size;
    +    }
    +
    +    private List<Object[]> size = new LinkedList<>();
    +
    +    public SlidingWindow slideBy(long slidingQuantity, Quantification.Unit slidingUnit)
    +    {
    +      SlidingWindow sw = new SlidingWindow(this.size);
    +      sw.delta.add(new Object[]{slidingQuantity, slidingUnit});
    +      return sw;
    +    }
    +
    +    public FixedWindow and(long quantity, Quantification.Unit unit)
    +    {
    +      size.add(new Object[]{quantity, unit});
    +      return this;
    +    }
    +
    +    public List<Object[]> getSize()
    +    {
    +      return size;
    +    }
    +  }
    +
    +  public static class SlidingWindow extends FixedWindow
    +  {
    +
    +    SlidingWindow(List<Object[]> size)
    +    {
    +      super(size);
    +    }
    +
    +    List<Object[]> delta = new LinkedList<>();
    +
    +    @Override
    +    public SlidingWindow and(long quantity, Quantification.Unit unit)
    +    {
    +      delta.add(new Object[]{quantity, unit});
    +      return this;
    +    }
    +  }
    +
    +  public static class SessionWindow extends WindowOption
    --- End diff --
    
    My point is that this is wholly unclear from the code itself :). The code is not self
explanatory and there's no clarifying documentation. This hinders long term maintenance and
support.
    
    
    
    Sent with Good (www.good.com)
    ________________________________
    From: David Yan <notifications@github.com>
    Sent: Thursday, June 9, 2016 4:59:03 PM
    To: apache/apex-malhar
    Cc: Ganelin, Ilya; Comment
    Subject: Re: [apache/apex-malhar] High-level WindowedStream API (Review only) (#316)
    
    
    In library/src/main/java/com/datatorrent/lib/window/WindowOption.java<https://github.com/apache/apex-malhar/pull/316#discussion_r66520504>:
    
    > +    SlidingWindow(List<Object[]> size)
    > +    {
    > +      super(size);
    > +    }
    > +
    > +    List<Object[]> delta = new LinkedList<>();
    > +
    > +    @Override
    > +    public SlidingWindow and(long quantity, Quantification.Unit unit)
    > +    {
    > +      delta.add(new Object[]{quantity, unit});
    > +      return this;
    > +    }
    > +  }
    > +
    > +  public static class SessionWindow extends WindowOption
    
    
    These concepts are detailed here:
    https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102
    http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf
    
    It's part of the windowing concept in the Apache Beam API.
    
    -
    You are receiving this because you commented.
    Reply to this email directly, view it on GitHub<https://github.com/apache/apex-malhar/pull/316/files/45d56b474567e24205c5e47db4bca3cce9ea5485#r66520504>,
or mute the thread<https://github.com/notifications/unsubscribe/ABaiUXyv_yLthWmls1Vc_hZD3BSVcPZJks5qKH6XgaJpZM4IxjTm>.
    ________________________________________________________
    
    The information contained in this e-mail is confidential and/or proprietary to Capital
One and/or its affiliates and may only be used solely in performance of work or services for
Capital One. The information transmitted herewith is intended only for use by the individual
or entity to which it is addressed. If the reader of this message is not the intended recipient,
you are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message