beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <j...@apache.org>
Subject [jira] [Commented] (BEAM-3073) Connect to Apache ignite via JdbcIO sdk
Date Wed, 18 Oct 2017 05:50:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208847#comment-16208847
] 

Jean-Baptiste Onofré commented on BEAM-3073:
--------------------------------------------

{{JdbcIO}} just relays on pure JDBC. So, if you provide a correct DataSource, it should work
with any backend. In your case, it seems that the connection can't be established by the datasource
(connection timeout).

Can you check:
1. that the IgniteJdbcDriver is in the classloader ?
2. that you can connect on localhost:11211 (no Windows firewall, or other binding) ?

> Connect to Apache ignite via JdbcIO sdk
> ---------------------------------------
>
>                 Key: BEAM-3073
>                 URL: https://issues.apache.org/jira/browse/BEAM-3073
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Rick Lin
>            Assignee: Jean-Baptiste Onofré
>            Priority: Minor
>
> Hi all,
> {color:#14892c}I tried to connect Apache Ignite(In-memory) via the beam's sdk:org.apache.beam.sdk.io.jdbc.JdbcIO
> Here, i am not sure if the JdbcIO sdk only is provided for some specific Database: MySQL(disk),
postgreSQL(disk)?{color}
> my java test code is as follows:
> import java.sql.PreparedStatement;
> import java.sql.SQLException;
> import java.util.ArrayList;
> import java.util.List;
> import org.apache.beam.sdk.Pipeline;
> import org.apache.beam.sdk.io.jdbc.JdbcIO;
> import org.apache.beam.sdk.options.PipelineOptionsFactory;
> import org.apache.beam.sdk.transforms.Create;
> import org.apache.beam.sdk.values.KV;
> import org.apache.beam.sdk.values.PCollection;
> public class BeamtoJDBC {
> 	public static void main(String[] args) {		
> 		Integer[] value=new Integer[] {1,2,3,4,5};
> 		List<KV<Integer, Integer>> dataList = new ArrayList<>();
> 		int n=value.length;
> 		int count=0;
> 		for (int i=0; i<n; i++)
> 		{
> 			dataList.add(KV.of(count,value[i]));
> 			count=count+1;			
> 		}
> 		
> 		Pipeline p = Pipeline.create(PipelineOptionsFactory.fromArgs(args).withValidation().create());
> 		
> 		PCollection<KV<Integer, Integer>> data=p.apply("create data with time",Create.of(dataList));
> 		data.apply(JdbcIO.<KV<Integer, Integer>>write()
> 				.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration
> 						.create("org.apache.ignite.IgniteJdbcDriver", "jdbc:ignite://localhost:11211/")
> 						)		
> 				.withPreparedStatementSetter(new JdbcIO.PreparedStatementSetter<KV<Integer,
Integer>>() {
> 					public void setParameters(KV<Integer, Integer> element, PreparedStatement
query)
> 						throws SQLException {
> 						query.setInt(1, element.getKey());
> 						query.setInt(2, element.getValue());
> 					}
> 				})
> 			);
> 		p.run();
> 	}
> }
> {color:#d04437}my error message is: 
> " InvocationTargetException: org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException:
Cannot create PoolableConnectionFactory 
> (Failed to establish connection.): Failed to get future result due to waiting timed out.
"{color}
> {color:#14892c}I would like to know whether the connection between beam and ignite is
feasible or not?{color}
> Thanks
> Rick



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message