nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toivo Adams (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (NIFI-322) New Database Connection Pooling Controller Service
Date Sat, 28 Mar 2015 17:52:53 GMT

     [ https://issues.apache.org/jira/browse/NIFI-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Toivo Adams updated NIFI-322:
-----------------------------
    Attachment: NIFI-322_28mar2015.patch

Thanks for help and patience.
Attached please find new patch.

1. It's little bit unusual but return to pool is here, no extra code is needed. When asking
connection, pool return wrapper which contain actual connection. And calling connection.close()
will return connection to pool, not closing it actually.
Please see testExhaustPool() and testGetManyNormal().
This gives ability to to old code which just take connection as input parameter and dont know
nothing about connection pool. Code just need to call usual connection.close()

2. testExternalJDBCDriverUsage() and testURLClassLoaderGetConnection() are marked with @Ignore
Maybe patch was still incorrect or you mean something more?

3. Right, "Database Type" is much better. Vendor is also good name, but for example Derby
has 2 versions, embedded and server, so vendor is not enough.

4. Removed default port.

5. JDBC vendors world is not straightforward. Often not only database vendors offer JDBC drivers
but other companies may also offer perfectly well working drivers with different driver names.
And even the same database vendor may offer different drivers with different names.
For example Sybase offer jConnect 6.0 and jConnect 7.0 and driver names are correspondingly
com.sybase.jdbc3.jdbc.SybDriver and com.sybase.jdbc4.jdbc.SybDriver
My idea was to provide default driver name which user can change.
Currently DatabaseSystems contain only few database types, in future this list should expanded
to help user.

6. My idea was to help user and describe some known database types. So DatabaseSystems contain
description like "MariaDB is a community-developed fork of the MySQL relational database management
system intended to remain free under the GNU GPL." and default driver name. But no actual
driver is included in NiFi distribution.
There is also section header in code “JDBC driver jar must be loaded from external location”
I hope this is acceptable?
I added also comments to DatabaseSystems code, just to be sure.

7. I created
/nifi-dbcp-service-bundle/nifi-dbcp-service-nar/src/main/resources/META-INF/NOTICE
	Not sure it's correct.

8. Max Total Connections – fixed.

9. Added nifi-dbcp-service dependency to dependencyManagement.
Added nifi-dbcp-service-nar dependency to nifi-assembly/pom.xml dependencies.
Maven started complaining 
Project build error: 'dependencies.dependency.version' for org.apache.nifi:nifi-dbcp-service-nar:nar
is missing.	pom.xml	/nifi-assembly	line 169	
So I added version also.

Added nifi-dbcp-service-bundle to nifi-standard-services/pom.xml 

Thanks
Toivo

> New Database Connection Pooling Controller Service
> --------------------------------------------------
>
>                 Key: NIFI-322
>                 URL: https://issues.apache.org/jira/browse/NIFI-322
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Toivo Adams
>            Assignee: Mark Payne
>            Priority: Minor
>              Labels: controller
>         Attachments: DBCPServiceApacheDBCP14.java, DBCPServiceTest.java, DatabaseSystemDescriptor.java,
DatabaseSystems.java, NIFI-322_01mar2015.patch, NIFI-322_03mar2015.patch, NIFI-322_08mar2015.patch,
NIFI-322_14mar2015.patch, NIFI-322_22mar2015.patch, NIFI-322_24mar2015.patch, NIFI-322_26mar2015.patch,
NIFI-322_28mar2015.patch, TestDatabaseSystems.java, TestProcessor.java
>
>
> Often DataFlows contain many processors which deal with database - select, update or
delete different data in different tables. 
> Yet database is same and connection pooling helps to speed up connecting to database
(open connection is fairly expensive). Also configuration must be done only in one place.

> Database Connection Pooling Controller Service helps to solve this in consistent way.

> related
> https://issues.apache.org/jira/browse/NIFI-293 : Add a JDBC Processor for executing arbitrary
SQL queries. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message