flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alpinegizmo <...@git.apache.org>
Subject [GitHub] flink pull request #4020: [FLINK-6747] [table] [docs] Time attributes sectio...
Date Tue, 30 May 2017 16:53:51 GMT
Github user alpinegizmo commented on a diff in the pull request:

    --- Diff: docs/dev/table/streaming.md ---
    @@ -39,17 +39,293 @@ Dynamic table
     Time Attributes
    -### Event-time
    +Flink supports different notions of *time* in streaming programs.
    -* DataStream: Timestamps & WMs required, `.rowtime` (replace attribute or extend
    -* TableSource: Timestamps & WMs & DefinedRowtimeAttribute
    +- *Processing time* refers to the system time of the machine (also known as "wall-clock
time") that is executing the respective operation.
    +- *Event time* is the time that each individual event occurred on its producing device.
    +- *Ingestion time* is the time that events enter Flink, internally, it is treated similar
to event time.
    -{% top %}
    +For more information about time handling in Flink, see the introduction about [Event
Time and Watermarks]({{ site.baseurl }}/dev/event_time.html).
    +Table programs assume that a corresponding time characteristic has been specified for
the streaming environment:
    +<div class="codetabs" markdown="1">
    +<div data-lang="java" markdown="1">
    +{% highlight java %}
    +final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    +env.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime); // default
    +// alternatively:
    +// env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);
    +// env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
    +{% endhighlight %}
    +<div data-lang="scala" markdown="1">
    +{% highlight scala %}
    +val env = StreamExecutionEnvironment.getExecutionEnvironment
    +env.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime) //default
    +// alternatively:
    +// env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime)
    +// env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
    +{% endhighlight %}
    +Time-based operations such as [windows]({{ site.baseurl }}/dev/table/tableApi.html) in
both the Table API and SQL require information about the notion of time and its origin. Therefore,
tables can offer *logical time attributes* for indicating time and accessing corresponding
timestamps in table programs.
    +Time attributes can be part of every table schema. They are defined when creating a table
from a `DataStream` or pre-defined when using a `TableSource`. Once a time attribute is defined
at the beginning, it can be referenced as field and used in time-based operations.
    +As long as a time attribute is not modified and simply forwarded from one part of the
query to another, it remains a valid time attribute. Time attributes behave like regular timestamps
and can be accessed for calculations. If a time attribute is used in a calculation, it will
be materialized and becomes a regular timestamp. Regular timestamps do not cooperate with
Flink's time and watermarking system and can thus not be used for time-based operations anymore.
     ### Processing time
    -* DataStream: `.proctime` (only extend schema)
    -* TableSource: DefinedProctimeAttribute
    +Processing time allows a table program to produce results based on the time of the local
machine. It is the simplest notion of time but does not provide determinism. It does neither
require timestamp extraction nor watermark generation.
    --- End diff --
    It requires neither timestamp extraction nor watermark generation.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message