sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kate Ting <k...@cloudera.com>
Subject Re: [sqoop-user] exception with mysql import
Date Mon, 17 Oct 2011 17:00:35 GMT
[Please use sqoop-user@incubator.apache.org for faster response.]

Hi, a connection failure may occur if the Sqoop user does not have the
necessary permissions to access his/her MySQL database over the
network.

The following steps (in order) can resolve this:

1. Verify that you can connect to the database from the node where you
are running Sqoop:
$ mysql --host=<IP Address> --database=test --user=<username>
--password=<password>
If this works, it rules out any problem with network configuration of
the client or security/authentication configuration.

2. Add the network port for the server to your my.cnf file
(/etc/my.cnf), something like the following:
[mysqld]
...
port = xxxx
Set up a user account to connect via Sqoop.

3. Grant that user the necessary permissions to access the database
over the network:
Log into MySQL as root (mysql -u root -p<ThisIsMyPassword>)
Issue the following command:
mysql> grant all privileges on test.* to 'testuser'@'%' identified by
'testpassword'
(Replace testuser/testpassword with appropriate values.)

4. Note that doing this will enable the testuser to connect to the
MySQL server from any IP address. While this will work, it is not
advisable for a production environment. We advise consulting with your
DBA to grant the necessary privileges based on the setup topology.

5. If the database server's IP address changes, unless it is bound to
a static hostname in your server, the connect string passed into Sqoop
will also need to be changed.

Regards, Kate

On Sun, Oct 16, 2011 at 11:43 PM, foolbear <foolbeargm@gmail.com> wrote:
> When I import a table from mysql, there is an exception like this.
> This is the problem of sqoop or mysql ? Is some particular
> configuration needed for mysql ?
>
>
> 11/10/17 14:16:25 INFO mapred.JobClient: Running job:
> job_201109231031_0204
> 11/10/17 14:16:26 INFO mapred.JobClient:  map 0% reduce 0%
> 11/10/17 14:16:43 INFO mapred.JobClient:  map 10% reduce 0%
> 11/10/17 14:16:44 INFO mapred.JobClient:  map 30% reduce 0%
> 11/10/17 14:16:48 INFO mapred.JobClient:  map 50% reduce 0%
> 11/10/17 14:16:51 INFO mapred.JobClient:  map 60% reduce 0%
> 11/10/17 14:16:52 INFO mapred.JobClient:  map 70% reduce 0%
> 11/10/17 14:16:55 INFO mapred.JobClient:  map 90% reduce 0%
> 11/10/17 14:17:00 INFO mapred.JobClient:  map 100% reduce 0%
> 11/10/17 14:26:50 INFO mapred.JobClient: Task Id :
> attempt_201109231031_0204_m_000004_0, Status : FAILED
> java.io.IOException: SQLException in nextKeyValue
>        at
> com.cloudera.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:
> 245)
>        at org.apache.hadoop.mapred.MapTask
> $NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
>        at
> org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:
> 67)
>        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>        at
> com.cloudera.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:
> 187)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:
> 1127)
>        at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure
>
> The last packet successfully received from the server was 79
> milliseconds ago.  The last packet sent successfully to the server was
> 602,726 milliseconds ago.
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:
> 39)
>        at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:
> 27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
>        at
> com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:
> 1118)
>        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3055)
>        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2941)
>        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3489)
>        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
>        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1416)
>        at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:418)
>        at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:397)
>        at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:7171)
>        at
> com.netease.backend.db.plan.operator.RSSimpleMergeOperator.getNextTuple(RSSimpleMergeOperator.java:
> 38)
>        at
> com.netease.backend.db.plan.SimpleSelectPlan.getNextTuple(SimpleSelectPlan.java:
> 43)
>        at
> com.netease.backend.db.plan.performer.QueryPlanPerformer.getNextTuple(QueryPlanPerformer.java:
> 72)
>        at com.netease.backend.db.DBResultSet.next(DBResultSet.java:186)
>        at
> com.cloudera.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:
> 234)
>        ... 11 more
> Caused by: java.io.EOFException: Can not read response from server.
> Expected to read 112 bytes, read 82 bytes before connection was
> unexpectedly lost.
>        at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2502)
>        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3008)
>        ... 23 more
>
> --
> NOTE: The mailing list sqoop-user@cloudera.org is deprecated in favor of Apache Sqoop
mailing list sqoop-user@incubator.apache.org. Please subscribe to it by sending an email to
incubator-sqoop-user-subscribe@apache.org.
>

Mime
View raw message