flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Re: Storm compatibility layer currently does not support Storm's SimpleJoin example
Date Fri, 29 May 2015 08:36:39 GMT
Hi Peter,

I started to look into the issue. However, I could not find the
following classes in the git repository:

org.apache.flink.stormcompatibility.util.AbstractStormSpout
org.apache.flink.stormcompatibility.util.OutputFormatter
org.apache.flink.stormcompatibility.util.StormBoltFileSink
org.apache.flink.stormcompatibility.util.StormBoltPrintSink
org.apache.flink.stormcompatibility.util.TupleOutputFormatter

Thus, I cannot compile and run the code. Can you please update the git.

However, I had a quick look into the code, and see a few issues right
away. For example the code uses "tuple.select(...)" what is not
supported so far. Right now, attributes can only be accessed via index.
Furthermore, the example uses a lot of meta information that cannot be
provided easily in Flink (ie, TopologyContext is only supported very
limited).

To add those things, I will need much more time. I don't think I should
be part of the first pull request, but be added later. I will integrate
the SingleJoinBolt example as a ITCase if the functionality is there. It
seems to be a good idea, to add more examples from storm-starter to
flink-storm-examples.


-Matthias


On 05/28/2015 09:37 AM, Szabó Péter wrote:
> Hi Matthias,
> 
> Of course, here is the package that contains the example's source classes.
> https://github.com/mbalassi/flink/tree/storm-backup/flink-staging/flink-streaming/flink-storm-examples/src/main/java/org/apache/flink/stormcompatibility/singlejoin
> It is mostly a copy-paste of SimpleJoin from storm-starter, though, I
> separated the spouts and the join bolt from the rest of the topology.
> I would be happy, if you could fix it. Probably I'm overlooking something.
> 
> Peter
> 
> 2015-05-27 17:13 GMT+02:00 Matthias J. Sax <mjsax@informatik.hu-berlin.de>:
> 
>> Hi Peter,
>>
>> Thanks a lot for your feedback. It's exiting to see, that somebody uses
>> the layer already. :)
>>
>> The current prototype is going to be merged soon. However, I am more
>> than happy to extend the functionality of the layer. Can you please
>> share your example with me, so I can see what the problem is and fix it?
>>
>> I am pretty sure, that the fix will be merged later on, too. There are
>> many other limitation in the layer. Right now, it is still in beta state.
>> ;)
>>
>> -Matthias
>>
>>
>> On 05/27/2015 03:48 PM, Szabó Péter wrote:
>>> Hey everyone,
>>>
>>> I experimented with the Storm compatibility layer Matthias wrote, and ran
>>> some Storm examples on Flink. I found that Storm's SimpleJoin example
>> does
>>> not work. I suppose it is because of the multiple input streams. I'm
>>> willing to add another example instead.
>>> Right now, I'm getting it through Aljoscha's streaming refactor.
>>>
>>> Peter
>>>
>>
>>
> 


Mime
View raw message