flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bowen Li <bowenl...@gmail.com>
Subject Re: Streaming write to Hive
Date Thu, 05 Sep 2019 04:16:15 GMT
Hi Qi,

With 1.9 out of shelf, I'm afraid not. You can make HiveTableSink
implements AppendStreamTableSink (an empty interface for now) so it can be
picked up in streaming job. Also, streaming requires checkpointing, and
Hive sink doesn't do that yet. There might be other tweaks you need to make.

It's on our list for 1.10, not high priority though.

Bowen

On Wed, Sep 4, 2019 at 2:23 AM Qi Luo <luoqi.bd@gmail.com> wrote:

> Hi guys,
>
> In Flink 1.9 HiveTableSink is added to support writing to Hive, but it
> only supports batch mode. StreamingFileSink can write to HDFS in streaming
> mode, but it has no Hive related functionality (e.g. adding Hive partition).
>
> Is there any easy way we can streaming write to Hive (with exactly-once
> guarantee)?
>
> Thanks,
> Qi
>

Mime
View raw message