flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: AWS Client Builder with default credentials
Date Thu, 20 Feb 2020 12:50:30 GMT
First things first, we do not intend for users to use anything in the S3 
filesystem modules except the filesystems itself,
meaning that you're somewhat treading on unsupported ground here.

Nevertheless, the S3 modules contain a large variety of AWS-provided 
CerentialsProvider implementations,
that can derive credentials from environment variables, system 
properties, files on the classpath and many more.

Ultimately though, you're kind of asking us how to use AWS APIs, for 
which I would direct you to the AWS documentation.

On 20/02/2020 13:16, David Magalhães wrote:
> I'm using 
> org.apache.flink.fs.s3base.shaded.com.amazonaws.client.builder.AwsClientBuilder 
> to create a S3 client to copy objects and delete object inside 
> a TwoPhaseCommitSinkFunction.
> Shouldn't be another way to set up configurations without put them 
> hardcoded ? Something like core-site.xml or flink-conf.yaml ?
> Right now I need to have them hardcoded like this.
> AmazonS3ClientBuilder.standard
>       .withPathStyleAccessEnabled(true)
>       .withEndpointConfiguration(
>         new EndpointConfiguration("http://minio:9000", "us-east-1")
>       )
>       .withCredentials(
>         new AWSStaticCredentialsProvider(new 
> BasicAWSCredentials("minio", "minio123"))
>       )
>       .build
> Thanks

View raw message