spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: [SQL][JDBC] Possible regression in JDBC reader
Date Fri, 25 Nov 2016 13:23:59 GMT
See https://github.com/apache/spark/pull/15499#discussion_r89008564 in
particular. Hyukjin / Xiao do we need to undo part of this change?

On Fri, Nov 25, 2016 at 1:02 PM Takeshi Yamamuro <linguin.m.s@gmail.com>
wrote:

> Hi,
>
> Seems we forget to pass `parts:Array[Partition]` into `JDBCRelation`.
> This was removed in this commit:
> https://github.com/apache/spark/commit/b3130c7b6a1ab4975023f08c3ab02ee8d2c7e995#diff-f70bda59304588cc3abfa3a9840653f4L237
>
> // maropu
>
> On Fri, Nov 25, 2016 at 9:50 PM, Maciej Szymkiewicz <
> mszymkiewicz@gmail.com> wrote:
>
> Hi,
>
> I've been reviewing my notes to https://git.io/v1UVC using Spark built
> from 51b1c1551d3a7147403b9e821fcc7c8f57b4824c and it looks like JDBC
> ignores both:
>
>    - (columnName, lowerBound, upperBound, numPartitions)
>    - predicates
>
> and loads everything into a single partition. Can anyone confirm that? It
> works just fine on 2.0.2 and before.
>
> --
> Best,
> Maciej
>
>
>
>
> --
> ---
> Takeshi Yamamuro
>

Mime
View raw message