beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1514) change default timestamp in KafkaIO
Date Wed, 01 Mar 2017 06:40:45 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889605#comment-15889605
] 

ASF GitHub Bot commented on BEAM-1514:
--------------------------------------

GitHub user XuMingmin opened a pull request:

    https://github.com/apache/beam/pull/2135

    BEAM-1514: change default timestamp in KafkaIO

    I use Java reflection here to check if timestamp is available and get the value, based
on the performance test as below. As the function is called for every record, it's worth to
choose a faster option.
    Direct: 1ns
    SpEL: 15ns
    Reflect: 5ns


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/XuMingmin/beam master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/2135.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2135
    
----
commit 9b0505ca5f244e28f8c83ca3f1037ae45e516791
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-22T00:40:16Z

    junit test for both Kafka 0.9 and 0.10

commit 07e207d845943c485e6280e1705cc7c153a2ef35
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-22T07:10:45Z

    add doc for how to test, and how it works

commit c1d74af9f69cca6d5a7a79a67ef184a13ea6b2f1
Author: XuMingmin <mmxu1986@gmail.com>
Date:   2017-02-22T17:58:29Z

    Merge pull request #1 from apache/master
    
    pull latest code

commit 71f83b7af6379155e30599f331ac39e68b046783
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-22T18:01:29Z

    JavaDoc style

commit 931027519be8f15999fef834609db6e2e63e087d
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-23T01:14:55Z

    revise to keep default kafka version as 0.9.0.1

commit d7093563067a31f268305bf001eb578e899745ec
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-24T22:12:21Z

    remove changes in TimestampTransform and WindowDoFnOperator

commit 280bc870568aff6825168a3faab3ed62870f6ee7
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-24T23:09:12Z

    revert TimestampTransform

commit 50d426be9b5cf7cc9d8c68a0a767a8cdcf8ab888
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-02-24T23:10:19Z

    revert TimestampTransform

commit d6ef256f7fee618512fccf622c0088d795a32bea
Author: XuMingmin <mmxu1986@gmail.com>
Date:   2017-03-01T06:12:28Z

    Merge pull request #2 from apache/master
    
    pull latest code from apache/beam

commit ad4c208992f1c8612bd319d82f6a28754032c9ad
Author: mingmxu <mingmxu@ebay.com>
Date:   2017-03-01T06:14:59Z

    use Java reflection for timestamp()

commit a95a9a74e08548977569f9b791b6d255468ef5f3
Author: XuMingmin <mmxu1986@gmail.com>
Date:   2017-03-01T06:23:01Z

    Merge pull request #3 from XuMingmin/BEAM-1514
    
    merge Beam 1514 to master

commit b496ffcfe2aac851ed9c163747edad1a1677fed8
Author: XuMingmin <mmxu1986@gmail.com>
Date:   2017-03-01T06:24:58Z

    Update pom.xml

commit 220b5af8172bee1319394c2cb4168bc6fbbc059e
Author: XuMingmin <mmxu1986@gmail.com>
Date:   2017-03-01T06:28:43Z

    Update ConsumerSpEL.java

----


> change default timestamp in KafkaIO
> -----------------------------------
>
>                 Key: BEAM-1514
>                 URL: https://issues.apache.org/jira/browse/BEAM-1514
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>            Reporter: Xu Mingmin
>            Assignee: Xu Mingmin
>
> When user use Kafka 0.10, the field 'timestamp' from Kafka should be used as the default
event timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message