hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eyal Golan <egola...@gmail.com>
Subject Re: instantiation of classes in MR
Date Fri, 30 Dec 2011 14:22:22 GMT
Great News !!
Thanks for the info.

So using reflection, I can inject different implementations of interfaces
(services) for the mapper (or reducer).
And this way I can test a mapper (or reducer).
Just by reflecting a stub instead of a real implementation.

Thanks,



Eyal Golan
egolan74@gmail.com

Visit: http://jvdrums.sourceforge.net/
LinkedIn: http://www.linkedin.com/in/egolan74
Skype: egolan74

P  Save a tree. Please don't print this e-mail unless it's really necessary



On Fri, Dec 30, 2011 at 2:50 PM, Harsh J <harsh@cloudera.com> wrote:

> Eyal,
>
> Yes, it is right to think of each Task attempt being one individual JVM
> running individually on any added Node. Multiple slots would mean multiple
> VMs in parallel as well. Yes, your use of reflection to build your objects
> will work just fine -- its all user-side java code that is executed.
>
> On 30-Dec-2011, at 4:42 PM, Eyal Golan wrote:
>
> Hi,
>
> I want to understand a basic concept in MR.
>
> If a mapper creates an instance of some class (using the 'new' operator),
> then the created class exists ONCE in the VM of this node.
> For each node.
> Correct?
>
> Now,
> what if instead of using the 'new' operator, the class is created using
> reflection.
> Is it valid in a MR?
> Will only one instance of the created class be existing in that node?
>
> Thanks,
>
>
> Eyal
>
> Eyal Golan
> egolan74@gmail.com
>
> Visit: http://jvdrums.sourceforge.net/
> LinkedIn: http://www.linkedin.com/in/egolan74
> Skype: egolan74
>
> P  Save a tree. Please don't print this e-mail unless it's really necessary
>
>
>

Mime
View raw message