flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: HBase 0.7.0 addon
Date Fri, 31 Oct 2014 12:14:27 GMT
For this I don't have time, we're working on upgrade HBase to 0.98 APIs
(and it's already working :))
However we should discuss about how to manage properly the version of hbase
and its hadoop dependencies..

Best,
Flavio

On Fri, Oct 31, 2014 at 11:32 AM, Kostas Tzoumas <ktzoumas@apache.org>
wrote:

> Agreed 100%.
>
> I created a JIRA for this:
> https://issues.apache.org/jira/browse/FLINK-1200
>
> Flavio, would you like to give it a go? Otherwise I will assign it to
> myself
>
> On Fri, Oct 31, 2014 at 10:12 AM, Flavio Pompermaier <pompermaier@okkam.it
> > wrote:
>
>> I think that a count operator is very useful for people wanting to run an
>> HelloWorld with Flink,
>> it's always the first test I do (and with Spark that is very easy..)
>>
>> Best,
>> Flavio
>>
>> On Fri, Oct 31, 2014 at 9:57 AM, Fabian Hueske <fhueske@apache.org>
>> wrote:
>>
>>> Hi Flavio,
>>>
>>> right now, there is no dedicated count operator in the API.
>>> You can do the work-around with appending a 1 and summing it up (see
>>> Wordcount example [1]).
>>> This is also what a dedicated count operator would internally do.
>>>
>>> It would be awesome to get some contributions for the HBase addon :-)
>>>
>>> Best, Fabian
>>>
>>> [1] http://flink.incubator.apache.org/docs/0.7-incubating/examples.html
>>>
>>> 2014-10-31 9:46 GMT+01:00 Flavio Pompermaier <pompermaier@okkam.it>:
>>>
>>>> We are trying to connect to HBase 0.98 so we'll probably contribute to
>>>> the HBase addon :)
>>>> Is there a count API for Dataset? What is the fastest way to run a
>>>> count on a dataset?
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>> On Fri, Oct 31, 2014 at 6:19 AM, Robert Metzger <rmetzger@apache.org>
>>>> wrote:
>>>>
>>>>> Okay, I've deployed the missing artifacts to maven central. it will
>>>>> take some hours until they are synchronized.
>>>>> The example in the "flink-hbase" module is still using the old Java
>>>>> API.
>>>>> But you should be able to use the Hbase Input format like this:
>>>>>         ExecutionEnvironment ee =
>>>>> ExecutionEnvironment.getExecutionEnvironment();
>>>>>         DataSet<Record> t = ee.createInput(new MyTableInputFormat());
>>>>>
>>>>> I think the Flink Hbase module is not very well-tested, so its likely
>>>>> that you'll find issues while using it.
>>>>>
>>>>> The only documentation on logging we have is this one:
>>>>> http://flink.incubator.apache.org/docs/0.7-incubating/internal_logging.html
>>>>>
>>>>> Are you only seeing the log messages from Flink or no messages at all?
>>>>>
>>>>>
>>>>> On Thu, Oct 30, 2014 at 4:10 PM, Flavio Pompermaier <
>>>>> pompermaier@okkam.it> wrote:
>>>>>
>>>>>> Ok thanks!I was trying to run a mapreduce flink job using an hbase
>>>>>> dataset but I wasn't able to make it run it locally. The one in the
addons
>>>>>> just specify a plan but it does not say how to test it.
>>>>>> Moreover I tried to put a log4j.properties in the classpath to debug
>>>>>> what's going on but I can't see any debug info. Do you have any hook/guide?
>>>>>> On Oct 30, 2014 11:58 PM, "Robert Metzger" <rmetzger@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> No, there is no reason for that. It actually seems like something
>>>>>>> went wrong while releasing Flink 0.7.0. I'll deploy the missing
artifacts.
>>>>>>>
>>>>>>> On Thu, Oct 30, 2014 at 9:26 AM, Flavio Pompermaier <
>>>>>>> pompermaier@okkam.it> wrote:
>>>>>>>
>>>>>>>> Hi to all,
>>>>>>>>
>>>>>>>> is there a reason why the 0.7.0 hbase addons is not deployed
on
>>>>>>>> maven central?
>>>>>>>>
>>>>>>>> Thanks in advance,
>>>>>>>> Flavio
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>
>>
>>
>> --
>>
>> Flavio Pompermaier
>>
>> *Development Department*_______________________________________________
>> *OKKAM**Srl **- www.okkam.it <http://www.okkam.it/>*
>>
>> *Phone:* +(39) 0461 283 702
>> *Fax:* + (39) 0461 186 6433
>> *Email:* pompermaier@okkam.it
>> *Headquarters:* Trento (Italy), via G.B. Trener 8
>> *Registered office:* Trento (Italy), via Segantini 23
>>
>> Confidentially notice. This e-mail transmission may contain legally
>> privileged and/or confidential information. Please do not read it if you
>> are not the intended recipient(S). Any use, distribution, reproduction or
>> disclosure by any other person is strictly prohibited. If you have received
>> this e-mail in error, please notify the sender and destroy the original
>> transmission and its attachments without reading or saving it in any manner.
>>
>>
>


-- 

Flavio Pompermaier

*Development Department*_______________________________________________
*OKKAM**Srl **- www.okkam.it <http://www.okkam.it/>*

*Phone:* +(39) 0461 283 702
*Fax:* + (39) 0461 186 6433
*Email:* pompermaier@okkam.it
*Headquarters:* Trento (Italy), via G.B. Trener 8
*Registered office:* Trento (Italy), via Segantini 23

Confidentially notice. This e-mail transmission may contain legally
privileged and/or confidential information. Please do not read it if you
are not the intended recipient(S). Any use, distribution, reproduction or
disclosure by any other person is strictly prohibited. If you have received
this e-mail in error, please notify the sender and destroy the original
transmission and its attachments without reading or saving it in any manner.

Mime
View raw message