flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jin Yi <eleanore....@gmail.com>
Subject Filter with large key set
Date Thu, 16 Jan 2020 06:48:10 GMT
Hi there,

I have the following usecase:
a key set say [A,B,C,....] with around 10M entries, the type of the entries
can be one of the types in BasicTypeInfo, e.g. String, Long, Integer etc...

and each message looks like below:
message: {
   header: A
   body: {}
}

I would like to use Flink to filter each message' header field, to see if
the value present in the key set.

*The key set needs to be dynamic, meaning at any time, we can perform
add/read/delete operations on the key set. *

Any suggestions are very welcome!

Thanks a lot!
Eleanore

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message