giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Cornell <m...@matthewcornell.org>
Subject Re: receiving messages that I didn't send
Date Thu, 25 Sep 2014 12:03:32 GMT
Thanks for replying, Pavan. I figured out that my message Writable (an
ArrayListWritable) needed to call clear() in readFields() before
calling super():

    @Override
    public void readFields(DataInput in) throws IOException {
        clear();
        super.readFields(in);
    }

This was an 'of course' moment when I realized it was, like other
Writables, being reused. But what I don't understand is why doesn't
ArrayListWritable#readFields() call clear? Isn't this a nasty bug? ...
Oh wait - sure enough:

ArrayListWritable object is not cleared in readFields()
https://issues.apache.org/jira/browse/GIRAPH-740

Thanks again,

matt


On Tue, Sep 23, 2014 at 11:46 AM, Pavan Kumar A <pavanka@outlook.com> wrote:
> Can you give more context?
> What are the types of messages, patch of your compute method, etc.
> You will not receive messages that are not sent, but one thing that can
> happen is
> -- message can have multiple parameters.
> suppose message objects can have 2 parameters
> m - a,b
> say in m's write(out) you do not handle the case of b = null
> m1 sets b
> m2 has b=null
> then because of incorrect code for m's write() m2 can show b = m1.b
> that is because message objects will be re-used when receiving. This is a
> Giraph gotcha, because of
> object reuse in most iterators.
>
> Thanks
>
>> From: matt@matthewcornell.org
>> Date: Tue, 23 Sep 2014 10:10:48 -0400
>> Subject: receiving messages that I didn't send
>> To: user@giraph.apache.org
>
>>
>> Hi Folks. I am refactoring my compute() to use a set of ids as its
>> message type, and in my tests it is receiving a message that it
>> absolutely did not send. I've debugged it and am at a loss.
>> Interestingly, I encountered this once before and solved it by
>> creating a copy of a Writeable instead of re-using it, but I haven't
>> been able to solve it this time. In general, does this anomalous
>> behavior indicate a Giraph/Hadoop gotcha'? It's really confounding!
>> Thank very much -- matt
>>
>> --
>> Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34
>> Dickinson Street, Amherst MA 01002 | matthewcornell.org



-- 
Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34
Dickinson Street, Amherst MA 01002 | matthewcornell.org

Mime
View raw message