flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Hi, question about orderBy two columns more
Date Mon, 02 Nov 2015 13:09:47 GMT
Hi Philip,

thanks for reporting the issue. I just verified the problem.
It is working correctly for the Java API, but is broken in Scala.

I will work on a fix and include it in the next RC for 0.10.0.

Thanks, Fabian

2015-11-02 12:58 GMT+01:00 Philip Lee <philjjoon@gmail.com>:

> Thanks for your reply, Stephan.
>
> So you said this is same as SQL
> but I got this result from this code. This is what we did not expect,
> right?
>
> val inputTuple = Seq((2,5),(2,3),(2,4),(3,2),(3,6))
>
> val outputTuple = env.fromCollection(inputTuple)
>   .sortPartition(0,Order.DESCENDING)
>   //.sortPartition(1,Order.ASCENDING)
>   .print()
>
> Output:
> (3,2)
> (3,6)
> (2,5)
> (2,3)
> (2,4)
>
> val inputTuple = Seq((2,5),(2,3),(2,4),(3,2),(3,6))
>
> val outputTuple = env.fromCollection(inputTuple)
>   .sortPartition(0,Order.DESCENDING)
>   .sortPartition(1,Order.ASCENDING)
>   .print()
>
> **
> Actual Output:
> (3,2)
> (2,3)
> (2,4)
> (2,5)
> (3,6)
> ‚Äč
> *Expected Output:*
> *(3,2)*
> *(3,6)*
> *(2,3)*
> *(2,4)*
> *(2,5)*
>
>
> Thanks,
> Phil
>
>
> On Mon, Nov 2, 2015 at 5:54 AM, Stephan Ewen <sewen@apache.org> wrote:
>
>> Actually, sortPartition(col1).sortPartition(col2) results in a single
>> sort that primarily sorts after col1 and secondarily sorts after col2, so
>> it is the same as in SQL when you state "ORDER BY col1, col2".
>>
>> The SortPartitionOperator created with the first "sortPartition(col1)"
>> call appends further columns, rather than instantiating a new sort.
>>
>> Greetings,
>> Stephan
>>
>>
>> On Sun, Nov 1, 2015 at 11:29 AM, Philip Lee <philjjoon@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I know when applying order by col, it would be
>>> sortPartition(col).setParralism(1)
>>>
>>> What about orderBy two columns more?
>>> If the sql is to state order by col_1, col_2,  sortPartition().
>>> sortPartition () does not solve this SQL.
>>>
>>> because orderby in sql is to sort the fisrt coulmn and the second column
>>> in the sorted first column. but for flink the funtion totally sorts each
>>> column.
>>>
>>> Any suggestion?
>>>
>>> Thanks,
>>> phil
>>>
>>>
>>
>

Mime
View raw message