Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (athena.apache.org: domain of ted.dunning@gmail.com
 designates 209.85.223.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <2433CB12-59AB-4A5C-9909-90526219F698@gmail.com>
References: <2433CB12-59AB-4A5C-9909-90526219F698@gmail.com>
From: Ted Dunning <ted.dunning@gmail.com>
Date: Tue, 18 Dec 2012 17:59:50 -0800
Message-ID: 
 <CAJwFCa3NYoNdQBwLcVLA-tOnupMHDqg+RTWvNZFV+FMP2WZX5A@mail.gmail.com>
Subject: Re: Is there anyway you could easily make a deep copy of Vector
 Writable class with hadoop's ReflectionUtils?
To: user@mahout.apache.org
Content-Type: multipart/alternative; boundary=14dae934069f07bbb604d12afa78

--14dae934069f07bbb604d12afa78
Content-Type: text/plain; charset=UTF-8

You can always just call clone() on the vector inside the VectorWritable.

On Tue, Dec 18, 2012 at 5:49 PM, Yunming Zhang
<zhangyunming1990@gmail.com>wrote:

> Hi,
>
> I have been trying to find a way to make a deep copy of key, value pairs
> inside SequenceFileRecordReader as I am implementing a getCurrentKeyCopy()
> and getCurrentValueCopy() method, but I am getting null pointer exception
> when calling "value.get()" in CIMapper.java, line 37,
>
> I want the record reader to create a new data object for the new key value
> pairs instead of reusing the original memory location,
>
> I found a few utility classes in Hadoop that should be able to make a deep
> copy or clone any writable class,
>
>        key = ReflectionUtils.copy(outer.getConfiguration(),
>                                    outer.getCurrentKey(), key);
>         value = ReflectionUtils.copy(conf, outer.getCurrentValue(), value);
>
> from MultiThreadedMapper and there is also WritableUtils.clone(...) method,
>
> However, both approaches seems to be failing for VectorWritable, I did
> notice that it is a bit different in that it wraps around another Mahout
> type, Vector, instead of a primitive type like int, long or string, in the
> cases of IntWritable and all that,
>
> I am not sure if this is why the copy or clone utilities in Hadoop
> designed for Writable is not working ?
>
> Currently it is giving me null pointer exception when I try to call
> value.get(), it does seem we copied something, but the vector inside
> VectorWritable might have failed to get copied,
>
> Thanks
>
> Yunming

--14dae934069f07bbb604d12afa78--