beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <>
Subject [jira] [Commented] (BEAM-621) Add MapValues and MapKeys functions
Date Thu, 29 Sep 2016 15:29:20 GMT


Jean-Baptiste Onofré commented on BEAM-621:

About the same idea, it would be great to have {{MapElements}} working with POJO. For instance
if the {{PCollection}} contains POJO like:

class MyPojo implements Serializable {
  public String getName() { ... }
  public Integer getAge() { ... }

today, to map this POJO to just name, we have to use {{MapElements}} with a function. It could
be interesting to provide a convenient function like {{MapElements.with("name"))}} that will
use reflection to map on {{getName()}}.

> Add MapValues and MapKeys functions
> -----------------------------------
>                 Key: BEAM-621
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Jean-Baptiste Onofré
> Currently, we have the {{MapElements}} {{PTransform}} that "convert" a {{PCollection}}
of {{KV}} to another {{PCollection}} (for instance {{String}}).
> A very classic mapping function is to just have the keys or values of {{KV}}.
> To do it currently, we can use {{MapElements}} or a generic {{ParDo}} (with {{DoFn}}).
> It would be helpful and reduce the user code to have {{MapValues}} and {{MapKeys}}. It
would take a {{PCollection}} of {{KV}}: {{MapKeys}} will map the input {{PCollection}} to
a {{PCollection}} of {{K}} and {{MapValues}} to a {{PCollection}} of {{V}}.

This message was sent by Atlassian JIRA

View raw message