At the moment I am developing a storm topology for processing raw machine measurement data. However, I am running into unexplainable problems with the spout.

I am running a simple storm topology on Azure HDInsight, written in Java. Events are read out of an eventhub, for which I am using the microsoft eventhub spout (version 0.9). This eventhub has 8 partitions, which means that I also need 8 instances of the Eventhubspout.

However, when I am running the topology for a couple of hours, the spouts stop receiving messages one after another, until each spout is quiet. No feedback is given whatsoever. The eventhub is still up and running when I inspect it through other means. Storm/the spouts are simply not registering anything anymore.

I have several ideas about what might be the problem here:

  1. Each partition only gets a message every +/- 4 seconds, which is ridiculously low for storm. Could it be that it automatically times-out and crashes because of this?
  2. Could the message 'sometimes' be too big, causing the spout to crash / show weird behavior?

For each of these mentioned reasons, shouldn't the eventhub spout automatically recover from this? What can be done to debug / tackle this problem?

Thank you very much for your time in advance!

Kind regards,