hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: Static state in Configuration and elsewhere
Date Thu, 11 Feb 2010 19:41:42 GMT
There are an enormous number of examples of the following line in user-side
code:

Configuration conf = new Configuration();

... This is going to need to still work transparently after any refactoring.
The new Configuration in this case needs to be populated with values from
the appropriate defaults files.

Maybe instead of using a singleton ConfigurationFactory (e.g.
ConfigurationFactory.getConfigFactory().newConfiguration()), you could
instead have Configuration's constructor use a service locator to get a
configuration "populator."

e.g:

class ConfigServiceLocator {
  static ConfigServiceLocator getServiceLocator(); /** this class is a
singleton */
  ConfigService getConfigService()
  void setConfigService(ConfigService);
}

class ConfigService {
  getAllThatStuffThatWasPreviouslyStaticState()
}

then in Configuration#Configuration() {

initWith(ConfigServiceLocator.getServiceLocator().getAllThatStuffThatWasPreviouslyStaticState());
}

Then there can be a static block which initializes the ConfigServiceLocator
with a default ConfigService instance that does everything as normal. For
testing, though, you could instantiate a new ConfigService, put whatever
state you want in it, and then update the ServiceLocator to use this new
instance instead.

Does that sound like a helpful alternative?
- Aaron





On Wed, Feb 10, 2010 at 3:09 AM, Steve Loughran <stevel@apache.org> wrote:

> Eric Sammer wrote:
>
>> All (notably commiters):
>>
>> In walking through the source, it seems that there are a number of cases
>> where static state exists. o.a.h.conf.Configuration is a good example of
>> a case were there is a static cache (see Configuration.REGISTRY) as well
>> as some class level state for default resources. These static members
>> make some testing scenarios difficult and I think they confuse and
>> complicate certain issues such as object lifecycle management.
>>
>> To that end, I started working on a patch to tease the registry
>> behavior, default resources, and other similar state and behavior into a
>> simple ConfigurationFactory that knows how to manufacture instances of
>> Configuration. For now, the Factory would act as a singleton but retains
>> a public constructor for simplifying testing and for what I hope is
>> easier integration with things like Spring down the road.
>>
>> While working on this, I realized this might be too big of a departure
>> from the original design. Is the static state thing on purpose? Is it
>> interesting to the rest of the community to factor this out? Does it
>> bother / disrupt anyone else? Are these sorts of design related changes
>> welcome?
>>
>
> I have looked at this in the past, a "what would it take to move to a
> config factory", and concluded it would be pretty tricky undertaking. There
> are lot of places that create new Configuration instances.
>
> That said, I would like, it, after I've got my lifecycle patches up to date
> and in.
>
> (I'd also like things like datanode and tasktracker to reread from config
> things like the namenode and JT addresses whenever they have to reconnect to
> the master nodes, but that's even more complex)
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message