flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: HBase 0.98 addon for Flink 0.8
Date Mon, 03 Nov 2014 08:55:52 GMT
There is no HBaseOutputFormat (and nothing equivalent) as far as I can see.
The only thing we had was the GenericTableOutputFormat which was
implemented against the deprecated Java Record API.
We would need to adapt the GenericTableOutputFormat to the new API.

2014-11-03 9:51 GMT+01:00 Fabian Hueske <fhueske@apache.org>:

> Ah, sorry. That's the one you removed ;-)
>
> 2014-11-03 9:51 GMT+01:00 Fabian Hueske <fhueske@apache.org>:
>
>> I'm not familiar with the HBase connector code, but are you maybe looking
>> for the GenericTableOutputFormat?
>>
>> 2014-11-03 9:44 GMT+01:00 Flavio Pompermaier <pompermaier@okkam.it>:
>>
>>> | was trying to modify the example setting hbaseDs.output(new
>>> HBaseOutputFormat()); but I can't see any HBaseOutputFormat class..maybe
>>> we
>>> shall use another class?
>>>
>>> On Mon, Nov 3, 2014 at 9:39 AM, Flavio Pompermaier <pompermaier@okkam.it
>>> >
>>> wrote:
>>>
>>> > Maybe that's something I could add to the HBase example and that could
>>> be
>>> > better documented in the Wiki.
>>> >
>>> > Since we're talking about the wiki..I was looking at the Java API (
>>> >
>>> http://flink.incubator.apache.org/docs/0.6-incubating/java_api_guide.html
>>> )
>>> > and the link to the KMeans example is not working (where it says For a
>>> > complete example program, have a look at KMeans Algorithm).
>>> >
>>> > Best,
>>> > Flavio
>>> >
>>> >
>>> > On Mon, Nov 3, 2014 at 9:12 AM, Flavio Pompermaier <
>>> pompermaier@okkam.it>
>>> > wrote:
>>> >
>>> >> Ah ok, perfect! That was the reason why I removed it :)
>>> >>
>>> >> On Mon, Nov 3, 2014 at 9:10 AM, Stephan Ewen <sewen@apache.org>
>>> wrote:
>>> >>
>>> >>> You do not really need a HBase data sink. You can call
>>> >>> "DataSet.output(new
>>> >>> HBaseOutputFormat())"
>>> >>>
>>> >>> Stephan
>>> >>> Am 02.11.2014 23:05 schrieb "Flavio Pompermaier" <
>>> pompermaier@okkam.it>:
>>> >>>
>>> >>> > Just one last thing..I removed the HbaseDataSink because I
think
>>> it was
>>> >>> > using the old APIs..can someone help me in updating that class?
>>> >>> >
>>> >>> > On Sun, Nov 2, 2014 at 10:55 AM, Flavio Pompermaier <
>>> >>> pompermaier@okkam.it>
>>> >>> > wrote:
>>> >>> >
>>> >>> > > Indeed this time the build has been successful :)
>>> >>> > >
>>> >>> > > On Sun, Nov 2, 2014 at 10:29 AM, Fabian Hueske <
>>> fhueske@apache.org>
>>> >>> > wrote:
>>> >>> > >
>>> >>> > >> You can also setup Travis to build your own Github
repositories
>>> by
>>> >>> > linking
>>> >>> > >> it to your Github account. That way Travis can build
all your
>>> >>> branches
>>> >>> > >> (and
>>> >>> > >> you can also trigger rebuilds if something fails).
>>> >>> > >> Not sure if we can manually trigger retrigger builds
on the
>>> Apache
>>> >>> > >> repository.
>>> >>> > >>
>>> >>> > >> Support for Hadoop 1 and 2 is indeed a very good addition
:-)
>>> >>> > >>
>>> >>> > >> For the discusion about the PR itself, I would need
a bit more
>>> time
>>> >>> to
>>> >>> > >> become more familiar with HBase. I do also not have
a HBase
>>> setup
>>> >>> > >> available
>>> >>> > >> here.
>>> >>> > >> Maybe somebody else of the community who was involved
with a
>>> >>> previous
>>> >>> > >> version of the HBase connector could comment on your
question.
>>> >>> > >>
>>> >>> > >> Best, Fabian
>>> >>> > >>
>>> >>> > >> 2014-11-02 9:57 GMT+01:00 Flavio Pompermaier <
>>> pompermaier@okkam.it
>>> >>> >:
>>> >>> > >>
>>> >>> > >> > As suggestes by Fabian I moved the discussion
on this mailing
>>> >>> list.
>>> >>> > >> >
>>> >>> > >> > I think that what is still to be discussed is
how  to
>>> retrigger
>>> >>> the
>>> >>> > >> build
>>> >>> > >> > on Travis (I don't have an account) and if the
PR can be
>>> >>> integrated.
>>> >>> > >> >
>>> >>> > >> > Maybe what I can do is to move the HBase example
in the test
>>> >>> package
>>> >>> > >> (right
>>> >>> > >> > now I left it in the main folder) so it will
force Travis to
>>> >>> rebuild.
>>> >>> > >> > I'll do it within a couple of hours.
>>> >>> > >> >
>>> >>> > >> > Another thing I forgot to say is that the hbase
extension is
>>> now
>>> >>> > >> compatible
>>> >>> > >> > with both hadoop 1 and 2.
>>> >>> > >> >
>>> >>> > >> > Best,
>>> >>> > >> > Flavio
>>> >>> > >>
>>> >>> > >
>>> >>> >
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message