ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: jdbc vs jdbc2 packages
Date Thu, 30 Mar 2017 23:06:48 GMT
Folks,

I had a chance to validate usability of both Ignite JDBC drivers at PGConf USA today trying
to connect to the cluster from the new JetBrains DataGrip SQL tool [1]. Together with JetBrains
folks we agreed on the following or spotted the issues listed below.

1) Default JDBC driver, that connects via a client node using a Spring XML config, is so counterintuitive
and twisted that JetBrains folks couldn’t set it up w/o my assistance. We force SQL tools
users, that accustomed to connect to DBs by hostname, to pass some weird Spring XML and add
up to 15 (!) jars to the classpath of a tool rather than one. This is huge +1 for the thin
client based driver that requires a hostname only. So, along with the thin client protocol
optimizations DML has to be supported for it as well [2]. 

2) Regardless of JDBC driver version if you execute a query like “SELECT * FROM PersonCache”
you’ll get a deserialization exception for _val that is implicitly added to the result set.
Luckily, this will be no longer the issue after this [3] improvement is released in 2.0.

3) It would be more user-friendly if the JDBC drivers are packaged under single “ignite-jdbc.jar”
and located in special directory of a release bundle. All the dependencies have to be in this
jar. The ticket is ready [4].

So I would encourage us to wrap up this discussion creating additional tickets in order to
renew the thin client based driver and promote it for tools and non transactional use cases.

[1] https://www.jetbrains.com/datagrip/ <https://www.jetbrains.com/datagrip/>
[2] https://issues.apache.org/jira/browse/IGNITE-4892 <https://issues.apache.org/jira/browse/IGNITE-4892>
[3] https://issues.apache.org/jira/browse/IGNITE-3487 <https://issues.apache.org/jira/browse/IGNITE-3487>
[4] https://issues.apache.org/jira/browse/IGNITE-4893 <https://issues.apache.org/jira/browse/IGNITE-4893>

—
Denis

> On Mar 30, 2017, at 1:09 AM, Denis Magda <dmagda@apache.org> wrote:
> 
> Val, Igniters,
> 
> I still believe the thin client has a right for living. It’s ideal for use cases when
someone attempts to connect to Ignite from a tool or some sort of interface and query the
data or update it in non transactional fashion. A TCP/IP address as a connection string to
the cluster is ideal for this scenario.
> 
> If someone decides to use JDBC programmatically and leverage from transactions then he
will switch to the JDBC versions that starts a client node with a passed Ignite configuration.
> 
> How do you like this?
> 
> In general, it sounds like a right movement to replace REST with more flexible and, probably,
unified protocol for thin client based JDBC implementation. But what if we extend REST a bit
(supporting pagination for SELECTs and DML) at the first phase so that the thin client driver
can be used right away in the scenarios I listed above? The rest of optimizations can be done
after that.
> 
> —
> Denis
> 
>> On Mar 23, 2017, at 7:24 PM, Valentin Kulichenko <valentin.kulichenko@gmail.com>
wrote:
>> 
>> I'm against keeping legacy thin client because:
>> 
>> - Having two ways to configure driver is unnecessary complication and very
>> bad from usability standpoint.
>> - It is much slower than client node, performance was the main driver
>> behind its deprecation.
>> 
>> What we should do, is improve usability of the client node, this will be
>> good improvement not only for JDBC driver. The list of required changes was
>> covered earlier in the thread, I will check if we have tickets for all of
>> them and provide a list.
>> 
>> -Val
>> 
>> On Thu, Mar 16, 2017 at 1:02 AM, Dmitriy Setrakyan <dsetrakyan@apache.org>
>> wrote:
>> 
>>> Denis, I completely agree that we should have a thin JDBC client. Can you
>>> file a ticket?
>>> 
>>> On Wed, Mar 15, 2017 at 4:58 PM, Denis Magda <dmagda@apache.org> wrote:
>>> 
>>>> Frankly, during these two a couple of day I had a chance to play with Web
>>>> Console and Ignite JDBC driver.
>>>> 
>>>> As many other users I referred to JDBC documentation in order to setup
>>> the
>>>> driver properly. However, then due to the recently reported issue I fell
>>>> back to JDBC v 1 (thin client based):
>>>> https://issues.apache.org/jira/browse/IGNITE-4829 <
>>>> https://issues.apache.org/jira/browse/IGNITE-4829>
>>>> 
>>>> I was amazed how swift JDBC v 1 and sluggish JDBC v 2 which is default.
>>>> Unfortunately, JDBC v 1 doesn’t support DML this is why I had hard time
>>>> finding out a workaround for IGNITE-4829. But, in any case I fully
>>> support
>>>> reincarnation of JDBC v 1.
>>>> 
>>>> *Val*, as one of the most experienced folks who participated in this
>>>> discussion, would you mind wrapping the discussion up in a form of a
>>> ticket
>>>> listing the design proposal?
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Feb 10, 2017, at 4:47 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
>>>> wrote:
>>>>> 
>>>>> I generally like the idea of supporting thin JDBC clients. Often it is
>>>> not
>>>>> feasible to start a client node on JDBC side.
>>>>> 
>>>>> On Fri, Feb 10, 2017 at 1:04 PM, Valentin Kulichenko <
>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>> 
>>>>>> Yakov,
>>>>>> 
>>>>>> There are two separate aspects that we can improve:
>>>>>> 
>>>>>> 1. Message routing as you described. I agree it can be useful in
some
>>>>>> scenarios and I definitely not against this feature, but honestly
I
>>>> haven't
>>>>>> seen a lot of requests for this so far.
>>>>>> 2. Server can initiate connection with client. This really blows
>>> users's
>>>>>> minds :)
>>>>>> 
>>>>>> I only meant that the latter is much more critical in my view.
>>>> Configuring
>>>>>> port forwarding on the cluster can be complicated, but it absolutely
>>>> makes
>>>>>> sense, while doing the same on client just sounds crazy. Client should
>>>> be
>>>>>> able to just connect, without having public IP and without additional
>>>>>> configuration on the router.
>>>>>> 
>>>>>> -Val
>>>>>> 
>>>>>> On Fri, Feb 10, 2017 at 1:58 AM, Yakov Zhdanov <yzhdanov@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> Val, can you please explain your statement. You suggest users
have
>>> port
>>>>>>> forwarding for dozens of nodes in their topologies so client
can
>>>> initiate
>>>>>>> connect to any of them? I don't think this is possible and routing
>>>> seems
>>>>>> to
>>>>>>> be the only possible option.
>>>>>>> 
>>>>>>> --Yakov
>>>>>>> 
>>>>>>> 2017-02-10 0:04 GMT+03:00 Valentin Kulichenko <
>>>>>>> valentin.kulichenko@gmail.com
>>>>>>>> :
>>>>>>> 
>>>>>>>> Yakov,
>>>>>>>> 
>>>>>>>> 1. Yes, I will file a ticket.
>>>>>>>> 
>>>>>>>> 4. I meant that server can currently initiate connection
with
>>> client,
>>>>>> and
>>>>>>>> that's the main problem here. Is there a way to avoid this?
Message
>>>>>>> routing
>>>>>>>> you're referring to can also be useful in some cases, but
much less
>>>>>>>> critical.
>>>>>>>> 
>>>>>>>> -Val
>>>>>>>> 
>>>>>>>> On Wed, Feb 8, 2017 at 9:20 PM, Yakov Zhdanov <
>>> yzhdanov@gridgain.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Val,
>>>>>>>>> 
>>>>>>>>> 1. Our clients should stop require persistent store implementation
>>> if
>>>>>>>> they
>>>>>>>>> do not need it. Can you please file a ticket? I know
you fixed some
>>>>>>>> places
>>>>>>>>> already. As an idea I would keep everything in binary
format until
>>> we
>>>>>>>>> really need it. Will that work?
>>>>>>>>> 2. We can try adding the very first step to fetch the
configuration
>>>>>> and
>>>>>>>>> then proceed with normal start.
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4675
>>>>>>>>> 3. Agree, but user needs to define the closures then.
I would think
>>>>>> on
>>>>>>>> how
>>>>>>>>> to put this to a product.
>>>>>>>>> 4. This needs to be implemented :) I mean we can communicate
to a
>>>>>>> client
>>>>>>>>> through server it is connected to.
>>>>>>>>> 
>>>>>>>>> Thanks!
>>>>>>>>> --
>>>>>>>>> Yakov Zhdanov, Director R&D
>>>>>>>>> *GridGain Systems*
>>>>>>>>> www.gridgain.com
>>>>>>>>> 
>>>>>>>>> 2017-02-09 5:19 GMT+07:00 Valentin Kulichenko <
>>>>>>>>> valentin.kulichenko@gmail.com
>>>>>>>>>> :
>>>>>>>>> 
>>>>>>>>>> Yakov,
>>>>>>>>>> 
>>>>>>>>>> I agree that investing in the legacy client doesn't
make sense -
>>>>>> it's
>>>>>>>>> slow
>>>>>>>>>> and outdated. Regarding your points:
>>>>>>>>>> 
>>>>>>>>>> 1. This is just another build step, but the JAR is
going to pretty
>>>>>>> fat
>>>>>>>> I
>>>>>>>>>> think (it will have to include Spring). Not ideal,
but definitely
>>>>>>>> better
>>>>>>>>>> than what we have now. However, our clients also
often require
>>> some
>>>>>>>> user
>>>>>>>>>> classes, like CacheStore implementations. This is
also a problem.
>>>>>>>>>> 
>>>>>>>>>> 2. That's a great idea! Actually, I'm not sure why
we require to
>>>>>> have
>>>>>>>>> full
>>>>>>>>>> verbose config on client that is consistent with
server. Why not
>>>>>>> fetch
>>>>>>>>> the
>>>>>>>>>> configuration from cluster during join? Not sure
how hard this
>>>>>> change
>>>>>>>> is,
>>>>>>>>>> but it can be a very big usability improvement. And
surely, JDBC
>>>>>>> driver
>>>>>>>>>> should be able to config with host:port without config
file.
>>>>>>>>>> 
>>>>>>>>>> 3. This can be already achieved with Compute Grid,
no? I don't
>>>>>> think
>>>>>>> we
>>>>>>>>>> need to add anything here.
>>>>>>>>>> 
>>>>>>>>>> Another issue with clients is that they currently
can't work
>>> behind
>>>>>>> NAT
>>>>>>>>>> without additional config which is not very trivial
>>>>>>> (AddressResolver).
>>>>>>>> Is
>>>>>>>>>> it possible to avoid server->client connections
in communication
>>>>>> SPI?
>>>>>>>>>> 
>>>>>>>>>> -Val
>>>>>>>>>> 
>>>>>>>>>> On Tue, Feb 7, 2017 at 9:09 PM, Yakov Zhdanov <
>>> yzhdanov@apache.org
>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> "undeprecating" - lol :D
>>>>>>>>>>> Consider introducing @Un annotation which negates
all annotations
>>>>>>> on
>>>>>>>>> the
>>>>>>>>>>> same level and below.
>>>>>>>>>>> 
>>>>>>>>>>> I would probably agree with Val and Vova, but
adding features to
>>>>>>>>>>> thin-client seems questionable to me.
>>>>>>>>>>> 
>>>>>>>>>>> Is these possible:
>>>>>>>>>>> 1. avoid dependencies on client machine and require
>>>>>> ignite-jdbc.jar
>>>>>>>>> only
>>>>>>>>>>> (e.g. gathering dependencies inside the jar).
>>>>>>>>>>> 2. make it possible to provide just address and
port to send join
>>>>>>>>> request
>>>>>>>>>>> to without providing the entire IgniteConfiguration.
Client node
>>>>>>>> sends
>>>>>>>>>> join
>>>>>>>>>>> request to the cluster with flag that this is
jdbc-driver
>>>>>>> connection
>>>>>>>>> and
>>>>>>>>>>> server-side topology omits configuration validation
and forces
>>>>>>> client
>>>>>>>>> to
>>>>>>>>>>> set some properties if this is necessary (e.g.
CommunicationSpi
>>>>>>>>>>> implementation class and settings)
>>>>>>>>>>> 3. add possibility to offload complex reduce
processing to
>>>>>> server.
>>>>>>>>> Which
>>>>>>>>>>> may be very helpful for main client-server use
cases when clients
>>>>>>>> being
>>>>>>>>>> run
>>>>>>>>>>> on much weaker machines.
>>>>>>>>>>> 
>>>>>>>>>>> ?
>>>>>>>>>>> 
>>>>>>>>>>> --Yakov
>>>>>>>>>>> 
>>>>>>>>>>> 2017-02-07 14:30 GMT+07:00 Vladimir Ozerov <vozerov@gridgain.com
>>>>>>> :
>>>>>>>>>>> 
>>>>>>>>>>>> Big +1 to Val, not only about JDBC, but about
our overall
>>>>>>> approach
>>>>>>>> to
>>>>>>>>>>>> clients. Starting a node with "client=true"
is:
>>>>>>>>>>>> + Very reach feature set, which is cool
>>>>>>>>>>>> - Tons of dependencies
>>>>>>>>>>>> - Tons of threads
>>>>>>>>>>>> 
>>>>>>>>>>>> It would be very cool if we have a true thin
client with small
>>>>>>>> single
>>>>>>>>>>> JAR.
>>>>>>>>>>>> It should have:
>>>>>>>>>>>> - Failover
>>>>>>>>>>>> - Load-balance
>>>>>>>>>>>> - Optional server "stickyness"
>>>>>>>>>>>> 
>>>>>>>>>>>> Once all these things are in place we will
be able to provide
>>>>>> the
>>>>>>>>> same
>>>>>>>>>>> API
>>>>>>>>>>>> as in current client, but with predictable
behavior and memory
>>>>>>>>>> footprint.
>>>>>>>>>>>> For instance our current client is not well-suited
for running
>>>>>>>>>> map-reduce
>>>>>>>>>>>> (compute or SQL) because it moves large amount
of data and
>>>>>>>> processing
>>>>>>>>>> to
>>>>>>>>>>>> the client, which is potentially a slow desktop
machine.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Feb 7, 2017 at 3:44 AM, Valentin
Kulichenko <
>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> There are two implementations of JDBC
driver - based on
>>>>>> legacy
>>>>>>>> thin
>>>>>>>>>>>> client
>>>>>>>>>>>>> (jdbc package) and on client node (jdbc2).
The first one was
>>>>>>>>>> deprecated
>>>>>>>>>>>>> when we introduced the latter, but now
I tend to think that
>>>>>>> this
>>>>>>>>> was
>>>>>>>>>>> not
>>>>>>>>>>>> a
>>>>>>>>>>>>> right decision. Thin client driver provides
worse
>>>>>> performance,
>>>>>>>> but
>>>>>>>>>> it's
>>>>>>>>>>>>> much easier to use, never requires additional
dependencies
>>>>>> like
>>>>>>>>>> Spring
>>>>>>>>>>>> and
>>>>>>>>>>>>> can be used from any remote machine.
Probably we can consider
>>>>>>>>>>>> undeprecating
>>>>>>>>>>>>> it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -Val
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Feb 6, 2017 at 2:02 AM, Sergi
Vladykin <
>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We have 2 different packages: jdbc
and jdbc2. Everything in
>>>>>>>> jdbc
>>>>>>>>> is
>>>>>>>>>>>>>> deprecated. Because of that new features
like DML support
>>>>>>> were
>>>>>>>>> not
>>>>>>>>>>>> added
>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> This seems to cause some problems
to our users. Can someone
>>>>>>>>>> clarify,
>>>>>>>>>>>> did
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> deprecated these classes wrongly
and we have to continue
>>>>>>>>> developing
>>>>>>>>>>>> them
>>>>>>>>>>>>> or
>>>>>>>>>>>>>> what?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message