nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierre Villard (Jira)" <j...@apache.org>
Subject [jira] [Updated] (NIFI-7591) Allow PutS3Object to post to AWS Snowball
Date Fri, 10 Jul 2020 16:26:00 GMT

     [ https://issues.apache.org/jira/browse/NIFI-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pierre Villard updated NIFI-7591:
---------------------------------
    Fix Version/s: 1.12.0
       Resolution: Fixed
           Status: Resolved  (was: Patch Available)

> Allow PutS3Object to post to AWS Snowball
> -----------------------------------------
>
>                 Key: NIFI-7591
>                 URL: https://issues.apache.org/jira/browse/NIFI-7591
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Juan C. Sequeiros
>            Assignee: Peter Turcsanyi
>            Priority: Major
>             Fix For: 1.12.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> When posting using PutS3Object to AWS SNOWBALL [1]it fails with "
> In short it supports the AWS s3 API in a limited function [2]
> [1][https://aws.amazon.com/getting-started/projects/migrate-petabyte-scale-data/faq/]
> [2][https://docs.aws.amazon.com/snowball/latest/developer-guide/using-adapter.html]
> {code:java}
> Chunk encoding is not supported yet (Service: Amazon S3; Status Code: 501; Error Code:
NotImplemented; Request ID: null; S3 Extended Request ID: null)         at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520)    
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4705)      
  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4652)        
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1807)        
at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:504)      
  at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2212)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2180)
        at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:443)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
        at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)         at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748) {code}
> {code}
> AWS docs state this: [2]
> "If your solution uses the AWS SDK for Java version 1.11.0 or newer, you must use the
following S3ClientOptions" 
> {code:java}
> disableChunkedEncoding() – Indicates that chunked encoding is not supported with the
adapter.
> setPathStyleAccess(true) – Configures the adapter to use path-style access for all
requests.
> [1]https://aws.amazon.com/getting-started/projects/migrate-petabyte-scale-data/faq/
> [2]https://docs.aws.amazon.com/snowball/latest/developer-guide/using-adapter.html{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message