spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] HeartSaVioR edited a comment on issue #25853: [SPARK-21869][SS] Apply Apache Commons Pool to Kafka producer
Date Tue, 10 Dec 2019 13:26:36 GMT
HeartSaVioR edited a comment on issue #25853: [SPARK-21869][SS] Apply Apache Commons Pool to
Kafka producer
URL: https://github.com/apache/spark/pull/25853#issuecomment-564030756
 
 
   I just went through implementing new approach of SPARK-21869: please take a look at the
branch below, which I revise the cache logic for producer.
   
   https://github.com/HeartSaVioR/spark/tree/SPARK-21869-new-cache-on-top-of-reverting-SPARK-21869
   
   The actual diff is in this commit, https://github.com/HeartSaVioR/spark/commit/ae2b607089ee115f5993eacf26444c3266fef81f
which is on top of revert commit of this PR.
   
   To minimize the change I just took the existing class and object but in other side trying
to encapsulate, hence things may not seem to be beauty. (For example, I'd really like to avoid
having only object and don't expose a chance to inject anything for testing - that's why I
separated KafkaDataConsumer and pool implementations.) But it should be easier for us to review
the concept.
   
   Please let me know if the concept/direction looks good; if we feel this as good alternative
for SPARK-21869, I guess we can revert the commit first, and raise a PR for new approach afterwards.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message