flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christophe Salperwyck <christophe.salperw...@gmail.com>
Subject HBase Input Format for streaming
Date Mon, 06 Jun 2016 11:49:42 GMT
Hi all,

I am trying to read data from HBase and use the windows functions of Flink
streaming. I can read my data using the ExecutionEnvironment but not from
the StreamExecutionEnvironment.

Is that a known issue?

Are the inputsplits used in the streaming environment?

Here a sample of my code:

final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

@SuppressWarnings("serial")
final DataStreamSource<ANA> anaDS = env.createInput(new
TableInputFormat<ANA>() {
...
}

final WindowedStream<ANA, Tuple, TimeWindow> ws = anaDS.
assignTimestampsAndWatermarks(new xxxxAssignerWithPunctuatedWatermarks()).
keyBy(0).
timeWindow(Time.days(30), Time.days(30));

ws.sum(2).printToErr();
env.execute();

The error I get is:
Caused by: java.io.IOException: No table result scanner provided!
at
org.apache.flink.addons.hbase.TableInputFormat.nextRecord(TableInputFormat.java:103)

It seems the "Result" is not read for a first time before calling this
function.

I built a "StreamingTableInputFormat" as a temporary work around but let me
know if there is something I did wrong.

Thanks for everything, Flink is great!

Cheers,
Christophe

Mime
View raw message