flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vino yang <yanghua1...@gmail.com>
Subject Re: Split one dataset into multiple
Date Tue, 06 Nov 2018 06:33:15 GMT
Hi madan,

I think you need to hash partition your records.
Flink supports hash partitioning of data.
The operator is keyBy.
If the value of your tag field is enumerable, you can also use split/select
to achieve your purpose.

Thanks, vino.

madan <madan.yellanki@gmail.com> 于2018年11月5日周一 下午6:37写道:

> Hi,
> I have a custom iterator which gives data of multitple entities. For
> example iterator gives data of Department, Employee and Address. Record's
> entity type is identified by a field value. And I need to apply different
> set of operations on each dataset. Ex., Department data may have
> aggregations, Employee and Address data are simply joined together after
> some filteration.
> If I have different datasets for each entity type the job is easy. So I am
> trying to split incoming data to different datasets. What is the best
> possible way to achieve this ?
> *Iterator can be read only once.
> --
> Thank you,
> Madan.

View raw message