flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: HBase 0.7.0 addon
Date Fri, 31 Oct 2014 08:57:45 GMT
Hi Flavio,

right now, there is no dedicated count operator in the API.
You can do the work-around with appending a 1 and summing it up (see
Wordcount example [1]).
This is also what a dedicated count operator would internally do.

It would be awesome to get some contributions for the HBase addon :-)

Best, Fabian

[1] http://flink.incubator.apache.org/docs/0.7-incubating/examples.html

2014-10-31 9:46 GMT+01:00 Flavio Pompermaier <pompermaier@okkam.it>:

> We are trying to connect to HBase 0.98 so we'll probably contribute to the
> HBase addon :)
> Is there a count API for Dataset? What is the fastest way to run a count
> on a dataset?
>
> Best,
> Flavio
>
> On Fri, Oct 31, 2014 at 6:19 AM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> Okay, I've deployed the missing artifacts to maven central. it will take
>> some hours until they are synchronized.
>> The example in the "flink-hbase" module is still using the old Java API.
>> But you should be able to use the Hbase Input format like this:
>>         ExecutionEnvironment ee =
>> ExecutionEnvironment.getExecutionEnvironment();
>>         DataSet<Record> t = ee.createInput(new MyTableInputFormat());
>>
>> I think the Flink Hbase module is not very well-tested, so its likely
>> that you'll find issues while using it.
>>
>> The only documentation on logging we have is this one:
>> http://flink.incubator.apache.org/docs/0.7-incubating/internal_logging.html
>>
>> Are you only seeing the log messages from Flink or no messages at all?
>>
>>
>> On Thu, Oct 30, 2014 at 4:10 PM, Flavio Pompermaier <pompermaier@okkam.it
>> > wrote:
>>
>>> Ok thanks!I was trying to run a mapreduce flink job using an hbase
>>> dataset but I wasn't able to make it run it locally. The one in the addons
>>> just specify a plan but it does not say how to test it.
>>> Moreover I tried to put a log4j.properties in the classpath to debug
>>> what's going on but I can't see any debug info. Do you have any hook/guide?
>>> On Oct 30, 2014 11:58 PM, "Robert Metzger" <rmetzger@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> No, there is no reason for that. It actually seems like something went
>>>> wrong while releasing Flink 0.7.0. I'll deploy the missing artifacts.
>>>>
>>>> On Thu, Oct 30, 2014 at 9:26 AM, Flavio Pompermaier <
>>>> pompermaier@okkam.it> wrote:
>>>>
>>>>> Hi to all,
>>>>>
>>>>> is there a reason why the 0.7.0 hbase addons is not deployed on maven
>>>>> central?
>>>>>
>>>>> Thanks in advance,
>>>>> Flavio
>>>>>
>>>>
>>>>
>

Mime
View raw message