spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: Does Spark SQL uses Calcite?
Date Sun, 20 Aug 2017 08:40:13 GMT
Hi Jules,

I am looking to connect to Spark via JDBC so I can run Spark SQL queries
via JDBC but not use SPARK SQL to connect to other JDBC sources.

Thanks!

On Sat, Aug 19, 2017 at 5:54 PM, Jules Damji <dmatrix@comcast.net> wrote:

> Try this link to see how you may connect  https://docs.databricks.com/
> spark/latest/data-sources/sql-databases.html
>
> Cheers
> Jules
>
> Sent from my iPhone
> Pardon the dumb thumb typos :)
>
> On Aug 19, 2017, at 5:27 PM, kant kodali <kanth909@gmail.com> wrote:
>
> Hi Russell,
>
> I went through this https://jaceklaskowski.gitbooks.io/mastering-apache-
> spark/spark-sql-thrift-server.html and I am still a bit confused on what
> hive is doing in here ? Is there any example I can look at on how to talk
> to Spark using Spark SQL JDBC driver alone and not hive ?
>
> Thanks,
> kant
>
>
>
>
>
> On Sat, Aug 12, 2017 at 6:07 AM, Russell Spitzer <
> russell.spitzer@gmail.com> wrote:
>
>> You don't have to go through hive. It's just spark sql. The application
>> is just a forked hive thrift server.
>>
>> On Fri, Aug 11, 2017 at 8:53 PM kant kodali <kanth909@gmail.com> wrote:
>>
>>> @Ryan it looks like if I enable thrift server I need to go through hive.
>>> I was talking more about having JDBC connector for Spark SQL itself other
>>> words not going through hive.
>>>
>>> On Fri, Aug 11, 2017 at 6:50 PM, kant kodali <kanth909@gmail.com> wrote:
>>>
>>>> @Ryan Does it work with Spark SQL 2.1.1?
>>>>
>>>> On Fri, Aug 11, 2017 at 12:53 AM, Ryan <ryan.hd.ren@gmail.com> wrote:
>>>>
>>>>> the thrift server is a jdbc server, Kanth
>>>>>
>>>>> On Fri, Aug 11, 2017 at 2:51 PM, <kanth909@gmail.com> wrote:
>>>>>
>>>>>> I also wonder why there isn't a jdbc connector for spark sql?
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Aug 10, 2017, at 2:45 PM, Jules Damji <dmatrix@comcast.net>
wrote:
>>>>>>
>>>>>> Yes, it's more used in Hive than Spark
>>>>>>
>>>>>> Sent from my iPhone
>>>>>> Pardon the dumb thumb typos :)
>>>>>>
>>>>>> On Aug 10, 2017, at 2:24 PM, Sathish Kumaran Vairavelu <
>>>>>> vsathishkumaran@gmail.com> wrote:
>>>>>>
>>>>>> I think it is for hive dependency.
>>>>>> On Thu, Aug 10, 2017 at 4:14 PM kant kodali <kanth909@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Since I see a calcite dependency in Spark I wonder where Calcite
is
>>>>>>> being used?
>>>>>>>
>>>>>>> On Thu, Aug 10, 2017 at 1:30 PM, Sathish Kumaran Vairavelu <
>>>>>>> vsathishkumaran@gmail.com> wrote:
>>>>>>>
>>>>>>>> Spark SQL doesn't use Calcite
>>>>>>>>
>>>>>>>> On Thu, Aug 10, 2017 at 3:14 PM kant kodali <kanth909@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> Does Spark SQL uses Calcite? If so, what for? I thought
the Spark
>>>>>>>>> SQL has catalyst which would generate its own logical
plans, physical plans
>>>>>>>>> and other optimizations.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Kant
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>

Mime
View raw message