apex-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t..@apache.org
Subject apex-malhar git commit: APEXMALHAR-2305 updated documentation on SessionWindow to reflect the new proto-session window semantics
Date Thu, 20 Oct 2016 18:08:27 GMT
Repository: apex-malhar
Updated Branches:
  refs/heads/master 0f8442472 -> 1bd113921


APEXMALHAR-2305 updated documentation on SessionWindow to reflect the new proto-session window
semantics


Project: http://git-wip-us.apache.org/repos/asf/apex-malhar/repo
Commit: http://git-wip-us.apache.org/repos/asf/apex-malhar/commit/1bd11392
Tree: http://git-wip-us.apache.org/repos/asf/apex-malhar/tree/1bd11392
Diff: http://git-wip-us.apache.org/repos/asf/apex-malhar/diff/1bd11392

Branch: refs/heads/master
Commit: 1bd113921a0f25b7e09a84514f7020bce3dd9aba
Parents: 0f84424
Author: David Yan <david@datatorrent.com>
Authored: Wed Oct 19 18:13:21 2016 -0700
Committer: David Yan <david@datatorrent.com>
Committed: Thu Oct 20 11:06:29 2016 -0700

----------------------------------------------------------------------
 .../windowedOperator/session-windows-1.png      | Bin 16868 -> 15139 bytes
 .../windowedOperator/session-windows-2.png      | Bin 15823 -> 14238 bytes
 .../windowedOperator/session-windows-3.png      | Bin 13323 -> 11613 bytes
 .../windowedOperator/session-windows-4.png      | Bin 0 -> 15176 bytes
 docs/operators/windowedOperator.md              |  20 +++++++++++--------
 5 files changed, 12 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-1.png
----------------------------------------------------------------------
diff --git a/docs/operators/images/windowedOperator/session-windows-1.png b/docs/operators/images/windowedOperator/session-windows-1.png
index 505ad6f..78712e8 100644
Binary files a/docs/operators/images/windowedOperator/session-windows-1.png and b/docs/operators/images/windowedOperator/session-windows-1.png
differ

http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-2.png
----------------------------------------------------------------------
diff --git a/docs/operators/images/windowedOperator/session-windows-2.png b/docs/operators/images/windowedOperator/session-windows-2.png
index 2fee78e..19bfab0 100644
Binary files a/docs/operators/images/windowedOperator/session-windows-2.png and b/docs/operators/images/windowedOperator/session-windows-2.png
differ

http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-3.png
----------------------------------------------------------------------
diff --git a/docs/operators/images/windowedOperator/session-windows-3.png b/docs/operators/images/windowedOperator/session-windows-3.png
index 14f0ccd..e7810eb 100644
Binary files a/docs/operators/images/windowedOperator/session-windows-3.png and b/docs/operators/images/windowedOperator/session-windows-3.png
differ

http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-4.png
----------------------------------------------------------------------
diff --git a/docs/operators/images/windowedOperator/session-windows-4.png b/docs/operators/images/windowedOperator/session-windows-4.png
new file mode 100644
index 0000000..43838c8
Binary files /dev/null and b/docs/operators/images/windowedOperator/session-windows-4.png
differ

http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/windowedOperator.md
----------------------------------------------------------------------
diff --git a/docs/operators/windowedOperator.md b/docs/operators/windowedOperator.md
index 0549b89..021842e 100644
--- a/docs/operators/windowedOperator.md
+++ b/docs/operators/windowedOperator.md
@@ -81,36 +81,40 @@ windowedOperator.setWindowOption(new WindowOption.TimeWindows(Duration.standardM
 
 ### `SessionWindows`
 
-`SessionWindow`s have variable durations and are based on the key of the tuple. Each tuple
is assigned to exactly one window. It takes a duration parameter `minGap`, which specifies
the minimum time gap between two `SessionWindow`s of the same key. 
+`SessionWindow`s have variable durations and are based on the key of the tuple. Each tuple
is assigned to exactly one window. It takes a duration parameter `minGap`, which specifies
the minimum time gap between two tuples that belong to two *different* `SessionWindows` of
the same key. `minGap` is also the duration of the "proto-session" window for a single timestamp,
and it is the minimum duration of any session window.
 
 ```java
 // Setting a session window option with a minimum gap of one hour
 windowedOperator.setWindowOption(new WindowOption.SessionWindows(Duration.standardHours(1)));
 ```
 
-To ensure that no two `SessionWindow`s of the same key are less than `minGap` apart, upon
an arrival of a tuple, the `WindowedOperator` does the following checks:
+Upon arrival of a tuple, a proto-session window is created. A proto-session window for a
tuple is a temporary session window with begin timestamp being the tuple timestamp and the
duration being `minGap`.
 
-#### The new tuple can be assigned to an existing `SessionWindow` without change
+![](images/windowedOperator/session-windows-4.png)
+
+To ensure that there are no two tuples of the same key in different session windows that
are less than `minGap` apart, the `WindowedOperator` does the following checks:
+
+#### The proto-session window can be fit into an existing `SessionWindow` of the same key
without change
 
 The new tuple is simply applied to the state of the existing `SessionWindow`.
 
 ![](images/windowedOperator/session-windows-3.png)
 
-#### The arrival of the new tuple would result in a merge of the two existing session windows
+#### The proto-session window overlaps with two existing session windows of the same key
 
-A new Session Window is created with the merged state of the two existing `SessionWindow`s
of the same key, plus the new tuple. The two existing `SessionWindow`s will be deleted and
retraction triggers for the two deleted windows will be fired. (Please see [here](#triggers)
for details on `Trigger`s)
+A new Session Window is created with the merged state of the two existing `SessionWindow`s,
plus the new tuple. The two existing `SessionWindow`s will be deleted and retraction triggers
for the two deleted windows will be fired. (Please see [here](#triggers) for details on `Trigger`s)
 
 ![](images/windowedOperator/session-windows-1.png) 
 
-#### The arrival of the new tuple would result in an extension of an existing `SessionWindow`s
+#### The proto-session window overlaps with one existing session window of the same key
 
-A new `SessionWindow` is created with the state of the existing `SessionWindow`, plus the
new tuple, with a longer duration than the existing `SessionWindow` to cover the new tuple.
The existing `SessionWindow` will be deleted and a retraction trigger for the old window will
be fired.
+A new `SessionWindow` is created with the state of the existing `SessionWindow`, plus the
new tuple, with a longer duration than the existing `SessionWindow` and possibly an earlier
begin timestamp to cover the new tuple. The existing `SessionWindow` will be deleted and a
retraction trigger for the old window will be fired.
 
 ![](images/windowedOperator/session-windows-2.png) 
 
 #### All of the above checks return false
 
-A new `SessionWindow` is created for the new tuple.
+The proto-session window is in effect and the new tuple is assigned to that window.
 
 ## Timestamp Extractor
 


Mime
View raw message