spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yash Sharma <yash...@gmail.com>
Subject [Spark][Kinesis] Could I get some committer review on the pull request
Date Tue, 05 Sep 2017 07:29:18 GMT
Hi All,
I've been working on a pull request [1] to allow Spark read from a specific
timestamp from Kinesis. I have iterated the patch with the help of other
contributors and we think that its in a good state now.

This patch would save hours of crash recovery time for Spark while reading
off Kinesis. Kinesis suffers from Throttling issues unlike Kafka and hence
this patch would essentially reduce the amount of data requested from
Kinesis.

I would love to hear some thoughts from the committers and see if I can
work on any improvements.

1. https://github.com/apache/spark/pull/18029

Best Regards,
Yash

Mime
View raw message