oodt-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris A. Mattmann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OODT-483) NumberFormatException when using RSS service to view transfers for large files
Date Mon, 06 Aug 2012 14:13:02 GMT

    [ https://issues.apache.org/jira/browse/OODT-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429154#comment-13429154
] 

Chris A. Mattmann commented on OODT-483:
----------------------------------------

Ross, +1 this makes a ton of sense. Feel free to use Review Board http://reviews.apache.org/
to post up a review and link it to this JIRA issue. Brian Foster, Ricky, Sheryl, Tom, Paul
R. and me and others are all active consumers of Review Board reviews, so that would be a
great thing.

Review Board or not, +1 to commit this patch and this is a critical fix.

Thanks!

                
> NumberFormatException when using RSS service to view transfers for large files
> ------------------------------------------------------------------------------
>
>                 Key: OODT-483
>                 URL: https://issues.apache.org/jira/browse/OODT-483
>             Project: OODT
>          Issue Type: Bug
>          Components: file manager
>            Reporter: Ross Laidlaw
>            Assignee: Ross Laidlaw
>            Priority: Minor
>              Labels: gsoc2012
>             Fix For: 0.5
>
>         Attachments: OODT-483.rlaidlaw.2012-08-06.patch.txt
>
>
> I used File Manager to ingest a large file and then attempted to view the progress of
the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]).
 This caused a NumberFormatException as follows:
> {code}
> java.lang.NumberFormatException: For input string: "11100111001101000000000000"
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> java.lang.Long.parseLong(Long.java:422)
> java.lang.Long.parseLong(Long.java:468)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
> org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
> org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
> javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
> {code}
> From further testing I found that this exception occurs for files approximately 10MB
in size or larger.
> This exception occurs because the XmlRpcStructFactory class is attempting to set the
value of a long variable using a binary representation (i.e. treating the binary number as
a decimal number).  The size of the binary representation exceeds the maximum capacity of
the long because it is not being converted to a decimal representation.
> I traced the problem to the following methods in class XmlRpcStructFactory [2]:
> {code}
> public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus
status)
> {
>   Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
>   statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
>   statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
>   statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
>   return statusHash;
> }
> public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String,
Object> statusHash) 
> {
>   FileTransferStatus status = new FileTransferStatus();
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
>   status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
>   status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
> statusHash.get("fileRef")));
>   return status;
> }
> {code}
> In the getXmlRpcFileTransferStatus method shown above, the following line converts the
value returned by 'status.getBytesTransferred()' to a binary representation and stores it
as a String in the 'statusHash' Hashtable:
> {code}
>   statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
> {code}
> But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc,
it isn't assumed to be a binary number:
> {code}
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
> {code}
> For large files, the binary representation exceeds the capacity of a long.  In the example
above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated
as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).
> A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc,
as follows:
> {code}
>   status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(),
2));{code}
> I tested this out on a large file (over 100MB) and it seemed to solve the problem.  Would
this be an acceptable fix?  I'll attach a patch to this issue for reference.
> An alternative solution would be not to store a binary representation in the statusHash
Hashtable.  But I'm assuming there's a reason why it's converted to binary for the hash. 

> [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
> [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message