flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Re: Tuple
Date Sun, 02 Aug 2015 22:45:01 GMT
Thanks for the explanation!

As I mentioned before, Tuple0 might also be helpful for streaming. And I
guess I will need it for Storm compatibility layer, too. (I need to
double check, but Storm supports zero-attribute-tuples, too).

With regard to the information I collected during the discussion, I vote
for keeping Tuple0 in Flink core, and fix the serialization problem.
Should we have another JIRA for this? Or should I extend the existing
JIRA? (https://issues.apache.org/jira/browse/FLINK-2457)

-Matthias


On 08/03/2015 12:22 AM, Chesnay Schepler wrote:
> First of all, it was a really good idea to start a discussion about this.
> 
> So the general idea behind Tuple0 was this:
> 
> The Python API maps python tuples to flink tuples. Python can have empty
> tuples, so i thought "well duh, let's make a Tuple0 class!". What i did
> not wanna do is create some non-Tuple object to represent empty tuples,
> I'd rather have them treated the same, because it's less work and
> creates simpler code.
> 
> When transferring the plan to java, certain parameters for operations
> are tuples, which can be empty aswell.
> This is where the Tuple0 class is really useful, because these empty
> tuples go through the same logic as other tuples.
> This is also why i want to keep the class, at least in the python
> project, for now.
> 
> For the actual program execution, I need a new solution. Funny story,
> while writing this reply i noticed that the Python API can't handle
> Tuple0 at runtime aswell. ha...ha... -.-
> 
> Guess I now know what I'm working on next.
> 
> On 02.08.2015 21:24, Matthias J. Sax wrote:
>> Can you elaborate how and why Python used Tuple0? If it cannot be
>> serialized similar to regular Tuples, what is the usage in Python? Right
>> now it seems, as there is no special serialization code for Tuple0.
>>
>> I just want to understand the topic in detail.
>>
>> -Matthias
>>
>>
>> On 08/01/2015 03:38 PM, Stephan Ewen wrote:
>>> I think a Tuple0 cannot be implemented like the current tuples, at least
>>> with respect to runtime serialization.
>>>
>>> The system makes the assumption that it makes progress in consuming
>>> bytes
>>> when deserializing values. If a Tuple= never consumes data from the byte
>>> stream, this assumption is broken. It would need at least one marker
>>> byte.
>>> Then it effectively is a Tuple1<Byte> disgusing itself as a tuple0.
>>>
>>>
>>>
>>> On Sat, Aug 1, 2015 at 1:38 PM, Matthias J. Sax <
>>> mjsax@informatik.hu-berlin.de> wrote:
>>>
>>>> I just double checked. Scala does not have type Tuple0. IMHO, it would
>>>> be best to remove Tuple0 for consistency. Having Tuple types is for
>>>> consistency reason with Scala in the first place, right? Please give
>>>> feedback.
>>>>
>>>> -Matthias
>>>>
>>>>
>>>> On 08/01/2015 01:04 PM, Matthias J. Sax wrote:
>>>>> I see.
>>>>>
>>>>> I think that it might be useful to have Tuple0, because in rare cases,
>>>>> you only want to "notify" a downstream operators (taking about
>>>>> streaming) that something happened but there is no actual data to be
>>>>> processed. Furthermore, if Flink cannot deal with Tuple0 it should be
>>>>> removed completely for consistency IMHO.
>>>>>
>>>>> I will open a JIRA for it.
>>>>>
>>>>> -Matthias
>>>>>
>>>>> On 07/31/2015 10:44 PM, Chesnay Schepler wrote:
>>>>>> also, I'm not sure if I ever sent a Tuple0 through a program, it
>>>>>> could
>>>>>> be that the system freaks out.
>>>>>>
>>>>>> On 31.07.2015 22:40, Chesnay Schepler wrote:
>>>>>>> there's no specific reason. it was added fairly recently by me
>>>>>>> (mid of
>>>>>>> april), and you're most likely the second person to use it.
>>>>>>>
>>>>>>> i didn't integrate into all our tuple related stuff because,
well, i
>>>>>>> never thought anyone would actually need it, so i saved myself
the
>>>>>>> trouble.
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> is there any specific reason, why Tuple.getTupleClass(int
arity)
>>>>>>>> does
>>>>>>>> not support arity zero? There is a class Tuple0, but it cannot
be
>>>>>>>> generator by Tuple.getTupleClass(...). Is it a missing feature
(I
>>>> would
>>>>>>>> like to have it).
>>>>>>>>
>>>>>>>> -Matthias
>>>>>>>>
>>>>>>>
>>>>
> 


Mime
View raw message