flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Re: Tuple
Date Sun, 02 Aug 2015 23:01:07 GMT
Thanks for the advice about Tuple0.

I personally don't see any advantage in having "flink-tuple" project. Do
I miss anything about it? Furthermore, I am not sure if it is a good
idea the have too many too small projects.

On 08/03/2015 12:48 AM, Stephan Ewen wrote:
> Tuple0 would need special serialization and comparator logic. If that is
> given, I see no reason not to support it.
> There is BTW, the request to create a dedicated "flink-tuple" project, that
> only contains the tuple classes. Any opinions on that?
> On Mon, Aug 3, 2015 at 12:45 AM, Matthias J. Sax <
> mjsax@informatik.hu-berlin.de> wrote:
>> Thanks for the explanation!
>> As I mentioned before, Tuple0 might also be helpful for streaming. And I
>> guess I will need it for Storm compatibility layer, too. (I need to
>> double check, but Storm supports zero-attribute-tuples, too).
>> With regard to the information I collected during the discussion, I vote
>> for keeping Tuple0 in Flink core, and fix the serialization problem.
>> Should we have another JIRA for this? Or should I extend the existing
>> JIRA? (https://issues.apache.org/jira/browse/FLINK-2457)
>> -Matthias
>> On 08/03/2015 12:22 AM, Chesnay Schepler wrote:
>>> First of all, it was a really good idea to start a discussion about this.
>>> So the general idea behind Tuple0 was this:
>>> The Python API maps python tuples to flink tuples. Python can have empty
>>> tuples, so i thought "well duh, let's make a Tuple0 class!". What i did
>>> not wanna do is create some non-Tuple object to represent empty tuples,
>>> I'd rather have them treated the same, because it's less work and
>>> creates simpler code.
>>> When transferring the plan to java, certain parameters for operations
>>> are tuples, which can be empty aswell.
>>> This is where the Tuple0 class is really useful, because these empty
>>> tuples go through the same logic as other tuples.
>>> This is also why i want to keep the class, at least in the python
>>> project, for now.
>>> For the actual program execution, I need a new solution. Funny story,
>>> while writing this reply i noticed that the Python API can't handle
>>> Tuple0 at runtime aswell. ha...ha... -.-
>>> Guess I now know what I'm working on next.
>>> On 02.08.2015 21:24, Matthias J. Sax wrote:
>>>> Can you elaborate how and why Python used Tuple0? If it cannot be
>>>> serialized similar to regular Tuples, what is the usage in Python? Right
>>>> now it seems, as there is no special serialization code for Tuple0.
>>>> I just want to understand the topic in detail.
>>>> -Matthias
>>>> On 08/01/2015 03:38 PM, Stephan Ewen wrote:
>>>>> I think a Tuple0 cannot be implemented like the current tuples, at
>> least
>>>>> with respect to runtime serialization.
>>>>> The system makes the assumption that it makes progress in consuming
>>>>> bytes
>>>>> when deserializing values. If a Tuple= never consumes data from the
>> byte
>>>>> stream, this assumption is broken. It would need at least one marker
>>>>> byte.
>>>>> Then it effectively is a Tuple1<Byte> disgusing itself as a tuple0.
>>>>> On Sat, Aug 1, 2015 at 1:38 PM, Matthias J. Sax <
>>>>> mjsax@informatik.hu-berlin.de> wrote:
>>>>>> I just double checked. Scala does not have type Tuple0. IMHO, it
>>>>>> be best to remove Tuple0 for consistency. Having Tuple types is for
>>>>>> consistency reason with Scala in the first place, right? Please give
>>>>>> feedback.
>>>>>> -Matthias
>>>>>> On 08/01/2015 01:04 PM, Matthias J. Sax wrote:
>>>>>>> I see.
>>>>>>> I think that it might be useful to have Tuple0, because in rare
>> cases,
>>>>>>> you only want to "notify" a downstream operators (taking about
>>>>>>> streaming) that something happened but there is no actual data
to be
>>>>>>> processed. Furthermore, if Flink cannot deal with Tuple0 it should
>>>>>>> removed completely for consistency IMHO.
>>>>>>> I will open a JIRA for it.
>>>>>>> -Matthias
>>>>>>> On 07/31/2015 10:44 PM, Chesnay Schepler wrote:
>>>>>>>> also, I'm not sure if I ever sent a Tuple0 through a program,
>>>>>>>> could
>>>>>>>> be that the system freaks out.
>>>>>>>> On 31.07.2015 22:40, Chesnay Schepler wrote:
>>>>>>>>> there's no specific reason. it was added fairly recently
by me
>>>>>>>>> (mid of
>>>>>>>>> april), and you're most likely the second person to use
>>>>>>>>> i didn't integrate into all our tuple related stuff because,
>> i
>>>>>>>>> never thought anyone would actually need it, so i saved
myself the
>>>>>>>>> trouble.
>>>>>>>>>> Hi,
>>>>>>>>>> is there any specific reason, why Tuple.getTupleClass(int
>>>>>>>>>> does
>>>>>>>>>> not support arity zero? There is a class Tuple0,
but it cannot be
>>>>>>>>>> generator by Tuple.getTupleClass(...). Is it a missing
feature (I
>>>>>> would
>>>>>>>>>> like to have it).
>>>>>>>>>> -Matthias

View raw message