beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Derek Hao Hu <phoenixin...@gmail.com>
Subject Re: How to get the PublishTime of PubsubMessages?
Date Mon, 02 Oct 2017 16:01:10 GMT
Thanks Reinier! Is it easy to get the [MessageId](
http://googleapis.github.io/googleapis/java/proto-google-cloud-pubsub-v1/0.1.19/apidocs/com/google/pubsub/v1/PubsubMessage.html#getMessageId--)
of the PubsubMessages?

PS: If you are familiar with this part of the Beam implementation, can you
also point out where it gets the timestamp attribute? I'm trying to
understand the implementation as well. :)

Cheers,

Derek

On Sat, Sep 30, 2017 at 5:58 AM, Reinier Kip <rkip@bol.com> wrote:

> Hi Derek,
>
>
> The publish time is the element's timestamp, retrievable via
> DoFn.ProcessContext#timestamp, unless you've configured a timestamp
> attribute with PubsubIO.Read#withTimestampAttribute. As far as I know,
> payload, attributes and publish time is all you get. What other information
> from Google's PubsubMessage are you looking for?
>
>
> Reinier
> ------------------------------
> *From:* Derek Hao Hu <phoenixinter@gmail.com>
> *Sent:* 30 September 2017 03:18:13
> *To:* user@beam.apache.org
> *Subject:* How to get the PublishTime of PubsubMessages?
>
> Hi,
>
> ​I'm trying to use [PubsubIO](​https://beam.apache.org/documentation/sdks/
> javadoc/2.1.0/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.html) to read
> [PubsubMessage](https://beam.apache.org/documentation/sdks/
> javadoc/2.1.0/org/apache/beam/sdk/io/gcp/pubsub/PubsubMessage.html).
>
> I've noticed that the PubsubMessage class in Beam only exposes Attribute
> and Payload but the [PubSubMessage in Google Cloud](http://googleapis.
> github.io/googleapis/java/proto-google-cloud-pubsub-v1/
> 0.1.19/apidocs/com/google/pubsub/v1/PubsubMessage.html?is-external=true)
> exposes a lot more information like `getPublishTime()`.
>
> Is there a way for us to get the same amount information in Beam easily?
> --
> Derek Hao Hu
>
> Software Engineer | Snapchat
> Snap Inc.
>



-- 
Derek Hao Hu

Software Engineer | Snapchat
Snap Inc.

Mime
View raw message