hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9196) Modify BloomFilter read() and write() to address memory concerns
Date Thu, 17 Jan 2013 22:52:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13556693#comment-13556693

James commented on HADOOP-9196:

That would definitely work.  I think it violates good OO principles to an  extent (open/close
principle, encapsulation), so I was hoping for a more elegant solution, but in this case I
suppose it's worth the trade-off.  
> Modify BloomFilter read() and write() to address memory concerns
> ----------------------------------------------------------------
>                 Key: HADOOP-9196
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9196
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: James
>            Priority: Minor
> It appears that org.apache.hadoop.util.bloom.BloomFilter's write() method creates a byte
array large enough to fit the entire bit vector into memory during serialization.  This is
unnecessary and may cause out of memory issues if the bit vector is sufficiently large and
memory is tight.   

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message