flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From glaksh100 <...@git.apache.org>
Subject [GitHub] flink pull request #6300: [FLINK-9692][Kinesis Connector] Adaptive reads fro...
Date Thu, 12 Jul 2018 19:46:20 GMT
Github user glaksh100 commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6300#discussion_r202156901
  
    --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java
---
    @@ -330,4 +347,24 @@ private GetRecordsResult getRecords(String shardItr, int maxNumberOfRecords)
thr
     	protected static List<UserRecord> deaggregateRecords(List<Record> records,
String startingHashKey, String endingHashKey) {
     		return UserRecord.deaggregate(records, new BigInteger(startingHashKey), new BigInteger(endingHashKey));
     	}
    +
    +	/**
    +	 * Adapts the maxNumberOfRecordsPerFetch based on the current average record size
    +	 * to optimize 2 Mb / sec read limits.
    +	 *
    +	 * @param averageRecordSizeBytes
    +	 * @return adaptedMaxRecordsPerFetch
    +	 */
    +
    +	private int getAdaptiveMaxRecordsPerFetch(long averageRecordSizeBytes) {
    --- End diff --
    
    Makes sense. Done.


---

Mime
View raw message