cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wido den Hollander (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-8640) Uploads to S3 Secondary Storage fail, stay at 0% completed
Date Fri, 17 Jul 2015 08:47:04 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631030#comment-14631030
] 

Wido den Hollander commented on CLOUDSTACK-8640:
------------------------------------------------

putObjectRequest.setProgressListener(new ProgressListener() {
    @Override
    public void progressChanged(ProgressEvent progressEvent) {
        // s_logger.debug(progressEvent.getBytesTransfered()
        // + " number of byte transferd "
        // + new Date());
        totalBytes += progressEvent.getBytesTransfered();
        if (progressEvent.getEventCode() == ProgressEvent.COMPLETED_EVENT_CODE) {
            s_logger.info("download completed");
            status = TemplateDownloader.Status.DOWNLOAD_FINISHED;
        } else if (progressEvent.getEventCode() == ProgressEvent.FAILED_EVENT_CODE) {
            status = TemplateDownloader.Status.UNRECOVERABLE_ERROR;
        } else if (progressEvent.getEventCode() == ProgressEvent.CANCELED_EVENT_CODE) {
            status = TemplateDownloader.Status.ABORTED;
        } else {
            status = TemplateDownloader.Status.IN_PROGRESS;
        }
    }
});


The new AWS SDK throws another event AFTER COMPLETED_EVENT_CODE. This makes the code fall
into the "ELSE" and set the download to IN_PROGRESS again.

However, getBytesTransfered() also returns 0 continuesly, so it will keep saying that the
upload is a 0 percent.

> Uploads to S3 Secondary Storage fail, stay at 0% completed
> ----------------------------------------------------------
>
>                 Key: CLOUDSTACK-8640
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8640
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: SystemVM
>    Affects Versions: Future, 4.5.1
>         Environment: Ceph RADOS Gateway with Civetweb as Secondary Storage
>            Reporter: Wido den Hollander
>            Assignee: Wido den Hollander
>              Labels: amazon, ceph, rados, s3, secondary_storage
>
> I noticed this after upgrading to 4.5.2 (build from 4.5 branch).
> Uploads never completed when a template was downloaded en directly uploaded to S3 secondary
storage provided by Ceph's RADOS Gateway using Multipart.
> After searching for hours I found this: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/event/ProgressEvent.html#getBytesTransferred()
> The ProgressEvent of the returned that 0 bytes had been transferred. But when using the
getBytes() method it actually works.
> The upload succeeds, but we check if the amount of uploaded bytes is equal or more then
what we expected. If not, we say the upload failed.
> This happens inside S3TemplateDownloader (which really needs some fixes btw....)
> Tracing this down if it's related to Ceph or actually something in S3TemplateDownloader.
> I also tried the Amazon SDK 1.9.34, but that didn't make a difference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message