hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jl...@streamy.com>
Subject RE: implementing selection/projection using mapreduce
Date Thu, 04 Dec 2008 17:18:22 GMT

There are probably a number of ways to do this, I am by no means a Java
expert, but if you create your Map class as a sub-class to the outer class
you should be able to access its members.  If that does not work, I know
that you can actually make your outer class and mapper class the same class.
In either case, our common practice is to always have the Map and Reduce
classes within the outer class so we can have global static variables for
cases exactly as you describe.


> -----Original Message-----
> From: abhinit [mailto:abhinit.kumar@gmail.com]
> Sent: Thursday, December 04, 2008 3:45 AM
> To: hbase-user@hadoop.apache.org
> Subject: implementing selection/projection using mapreduce
> Hi,
> I am implementing basic selection/projection using mapreduce on an
> HBase
> table.
> I have an outer class SelectProject which implements the tool interface
> and
> my mapper class,
> SelectProjectOperator extends TableMap.
> In my run method, I am parsing the command line argument for things
> like
> column names
> and predicate condition(say key = 10000 etc.) and I have to use these
> values
> inside the map
> method of my mapper class. All the variables to be used in the map
> function
> for selection/projection
> are static variables of the enclosing outer class(SelectProject). I et
> these
> variables in run method
> after parsing the comman-line arguments.
> However, in the map method I get a NullPointerException because non-of
> those
> static member variables
> are created. I understand this is not exactlt an HBase related issue
> and has
> something to do with java inner classes.
> But I thought this can be a very common case in mapreduce programming
> (passing a condition variable from the
> outer class to enclosed mapper class' map method).
> Thanks a lot for help.
> -Abhinit
> --
> Abhinit Kumar
> MSE, Computer & Information Science
> Class of 2009
> University of Pennsylvania
> 1-215-796-5136

View raw message