kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-6020) Broker side filtering
Date Fri, 06 Oct 2017 21:26:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Matthias J. Sax updated KAFKA-6020:
    Labels: needs-kip  (was: )

> Broker side filtering
> ---------------------
>                 Key: KAFKA-6020
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6020
>             Project: Kafka
>          Issue Type: Improvement
>          Components: consumer
>            Reporter: Pavel Micka
>              Labels: needs-kip
> Currently, it is not possible to filter messages on broker side. Filtering messages on
broker side is convenient for filter with very low selectivity (one message in few thousands).
In my case it means to transfer several GB of data to consumer, throw it away, take one message
and do it again...
> While I understand that filtering by message body is not feasible (for performance reasons),
I propose to filter just by message key prefix. This can be achieved even without any deserialization,
as the prefix to be matched can be passed as an array (hence the broker would do just array
prefix compare).

This message was sent by Atlassian JIRA

View raw message