hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Smith (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-765) Adding basic Spring DI support to IndexConfiguration class.
Date Wed, 10 Sep 2008 20:23:44 GMT

     [ https://issues.apache.org/jira/browse/HBASE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ryan Smith updated HBASE-765:
-----------------------------

    Description: 
Spring can configure classes/object graphs via xml.  I am pretty much able to configure the
entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java.
So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to
add support so Spring could set all class variables needed for initialization in IndexConfiguration
without invoking addFromXML().  

Since the class IndexConfiguration already has setters and getters for almost all its members,
it's almost compliant for a spring configuration bean except one issue: no ability to configure
columnMap outside of calling addFromXML().  The easiest way i can figure is to allow a setter
for the column map and put any logic for checking the map integrity there.  By adding a few
methods to IndexConfiguration.java , it should solve the issue.




  was:
Spring can configure classes/object graphs via xml.  I am pretty much able to configure the
entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java.
So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to
add support so Spring could set all class variables needed for initialization in IndexConfiguration
without invoking addFromXML().  

Since the class IndexConfiguration already has setters and getters for almost all its members,
it's almost compliant for a spring configuration bean except one issue: no ability to configure
columnMap outside of calling addFromXML().  The easiest way i can figure is to allow a setter
for the column map and put any logic for checking the map integrity there.  By adding a few
methods to IndexConfiguration.java , it should solve the issue.

I will be submitting a patch for it here shortly.....

EDIT: after further review, there are 2 more classes that are using addFromXML()
IndexOutputFormat and IndexTableReduce use the xml string to configure the indexconfig object.
I will have to modify these 3 classes so I will be developing the patch over this week, maybe
next week too.

EDIT 2: It seems that the distributed nature of Hadoop's MapReduce api is getting in the way
of spring loading these classes.
Since a new instance of new IndexConfiguration() needs to be called remotely, (on each node
in the cluster running the MR job), it makes it hard to have spring create the class when
ultimately, Hadoop has to create the class for each task, etc.  

I am going to submit changes to hadoop issue 2866.
http://www.mail-archive.com/core-dev@hadoop.apache.org/msg18574.html
Or maybe create a new issue for it to get some basic setters/getters and constructors patched.

There is also the possibility of getting spring to support map reduce natively.  Ill ask them
about it, but for now, I would say that this should go on the back burner, or just keep punting
it down until remote objects can get created via factory interfaces instead of reflection
internally as it is now.


> Adding basic Spring DI support to IndexConfiguration class.
> -----------------------------------------------------------
>
>                 Key: HBASE-765
>                 URL: https://issues.apache.org/jira/browse/HBASE-765
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.16.0, 0.1.0, 0.1.1, 0.1.2, 0.1.3
>         Environment: n/a
>            Reporter: Ryan Smith
>            Priority: Minor
>             Fix For: 0.19.0, 0.2.2
>
>   Original Estimate: 0.33h
>  Remaining Estimate: 0.33h
>
> Spring can configure classes/object graphs via xml.  I am pretty much able to configure
the entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java.
So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to
add support so Spring could set all class variables needed for initialization in IndexConfiguration
without invoking addFromXML().  
> Since the class IndexConfiguration already has setters and getters for almost all its
members, it's almost compliant for a spring configuration bean except one issue: no ability
to configure columnMap outside of calling addFromXML().  The easiest way i can figure is to
allow a setter for the column map and put any logic for checking the map integrity there.
 By adding a few methods to IndexConfiguration.java , it should solve the issue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message