pinot-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-pinot] jackjlli opened a new pull request #5039: Close Kafka consumer before building or downloading immutable segment
Date Sat, 01 Feb 2020 19:27:40 GMT
jackjlli opened a new pull request #5039: Close Kafka consumer before building or downloading
immutable segment
URL: https://github.com/apache/incubator-pinot/pull/5039
 
 
   When realtime immutable segment needs to be built or downloaded, there is a short period
of time that the new consuming segment has already started the Kafka consumption before the
immutable segment finishes building or downloading. 
   This leads to a scenario that two unclosed consumers exist for the same partitionId at
the same time. And this will also cause the metric system having two consumers with the same
metric name.
   
   When an immutable segment needs to be built or downloaded, it means this segment has already
done the Kafka consumption. Thus, it's ok to close the Kafka consumer.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Mime
View raw message