samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Riccomini" <criccom...@apache.org>
Subject Review Request 13867: SAMZA-3
Date Tue, 27 Aug 2013 22:17:14 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13867/
-----------------------------------------------------------

Review request for samza.


Repository: samza


Description
-------

switch to linked blocking queue in blocking envelope map. updated kafka's message sink to
have a needsMoreMessages method that lets the broker proxy filter out topics and partitions
that don't need to be added to the fetch requests because their queue isn't empty. added some
tests to verify new behavior for broker proxy and blocking envelope map. all tests pass.


switch from array blocking queue to an unbounded linked blocking queue. upside is we won't
get deadlocks anymore. downside is that implementations must manage when to stop feeding messages
into the queue to prevent OOME.


make blocking map throw illegal state exception when queue is full.


Diffs
-----

  samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java 96f31488b93c275bf1c472a1bf7039dcf8d7b01d

  samza-api/src/test/java/org/apache/samza/util/TestBlockingEnvelopeMap.java 2aed4645d8a35efaaa041d8db30949090dcb0460

  samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala cb5015d2ae55bd0d391605bdccc5cc972c53c4c6

  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala bd7794a14a66154a3d83296a559b9ec4158f5628

  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemFactory.scala 13c5baae73768a64eaf3ec31d997f6f91a548d0a

  samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 71fae593d8772afe136d68a6cf9d4ef8c3d7ac1d

  samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala 15510ae60b92f3a520a0d9d9c5a7c999bbb0cb09


Diff: https://reviews.apache.org/r/13867/diff/


Testing
-------


Thanks,

Chris Riccomini


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message