flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vinay patil <vinay18.pa...@gmail.com>
Subject Re: Facing Issues while migrating to S3A
Date Sat, 06 Aug 2016 06:34:05 GMT
Hi Stefan,

The migration from S3n to S3A is working fine when I try it on cluster
(linux box), the issue is only with the local windows machine.

I am not clear on this ,Can you please elaborate:
*"The code of the native method should be contained in something like a
HADOOP_HOME\bin\hadoop.dll"*

I want to test it locally so that I don't have to run every-time on cluster

Regards,
Vinay Patil

On Fri, Aug 5, 2016 at 3:18 PM, Stefan Richter [via Apache Flink User
Mailing List archive.] <ml-node+s2336050n8338h30@n4.nabble.com> wrote:

> Hi,
>
> from the exception, it seems like some native library is missing in your
> classpath. The code of the native method should be contained in something
> like a HADOOP_HOME\bin\hadoop.dll.
>
> Best,
> Stefan
>
> > Am 05.08.2016 um 11:32 schrieb vinay patil <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=8338&i=0>>:
> >
> > Hi All,
> >
> > I am facing issues when I changed from S3n to S3A, I am using
> > hadoop-aws-2.7.2 and aws-java-sdk-1.7.4 as dependencies
> >
> > Getting this error : java.lang.UnsatisfiedLinkError:
> > org.apache.hadoop.io.nativeio.NativeIO$Windows.access
> > Tried to change the fs.s3.buffer.dir path but getting same error.
> >
> > According to the documentation when I try to provide values in these
> > properties, it does not work and gives the credential exception:
> >
> > <property>
> >  <name>fs.s3.awsAccessKeyId</name>
> >  <value></value>
> > </property>
> >
> > <property>
> >  <name>fs.s3.awsSecretAccessKey</name>
> >  <value></value>
> > </property>
> >
> > When I change these properties to fs.s3a.access.key and
> fs.s3a.secret.key it
> > accepts the credentials but gives the above exception.
> >
> > Can you please guide me the correct way of migrating from S3n to S3A
> >
> > P.S I have followed all the steps in the documentation.
> >
> >
> >
> > --
> > View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Facing-Issues-
> while-migrating-to-S3A-tp8337.html
> > Sent from the Apache Flink User Mailing List archive. mailing list
> archive at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/Facing-Issues-while-migrating-to-S3A-tp8337p8338.html
> To start a new topic under Apache Flink User Mailing List archive., email
> ml-node+s2336050n1h83@n4.nabble.com
> To unsubscribe from Apache Flink User Mailing List archive., click here
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
> .
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Facing-Issues-while-migrating-to-S3A-tp8337p8349.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Mime
View raw message