flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Gregor <konstantin.gre...@tngtech.com>
Subject Re: Web Dashboard reports 0 for sent/received bytes and records
Date Thu, 30 Mar 2017 15:16:51 GMT
Hi Mohammad,

that's normal. The source will never have a value other than zero in the
"Records received" column, only in the "Records sent" column. But since
your whole program is chained to one subtask, the source will never send
anything to another subtask, because there is none.
Same for the sink, it will only count values it receives from its
preceding subtask, but due to chaining, there is no preceding subtask.
Thus there will only be zeros in your dashboard.
You can add your own Accumulators or Metrics and see the number of
records in your operators through that, or you could also disable the
chaining (but chaining is a good thing for performance, so you probably
don't want to do that).

Hope that helps,
best regards

Konstantin

On 30.03.2017 16:48, Mohammad Kargar wrote:
> Hi,
> 
> I have a simple job where FlinkKafkaConsumer08 reads data from a topic
> (with 8 partitions) and a custom Sink implementation inserts the data
> into Accumulo database. I can verify that the data is getting inserted
> into the db by adding simple system.out.println(...) to the invoke
> method of the sink and also monitoring Accumulo stats. However Flink Web
> Dashboard reports 0 for sent/received bytes and records (see attached
> file for a snapshot). Any thoughts?
> 
> *Hadoop platform*: Hortonworks Data Platform 2.5
> *Flink*: flink-1.2.0-bin-hadoop27-scala_2.10
> 
> Thanks,
> Mohammad

-- 
Konstantin Gregor * konstantin.gregor@tngtech.com
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082

Mime
View raw message