Mailing-List: contact user-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Users List" <user@commons.apache.org>
Received-SPF: pass (nike.apache.org: domain of tbessie@meez.com designates
 74.125.149.75 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAC3rAA1DKh3w+4UQAjZrBZqXcaCDU-Luw5zjQASz3z3gCBADxA@mail.gmail.com>
References: 
 <CAC3rAA3Ck_rfMYnfTOjEMu8Gf-yo=Mmr=kO1z=geqsxsKf211g@mail.gmail.com>
	<4F5F08AF.7090309@free.fr>
	<CAC3rAA1ZXPVpv1RDAyQSM3z8-jfhXjkfN6eVKNwOMw5rKq178w@mail.gmail.com>
	<4F60FE94.3080802@oliver-heger.de>
	<CAC3rAA1DKh3w+4UQAjZrBZqXcaCDU-Luw5zjQASz3z3gCBADxA@mail.gmail.com>
Date: Wed, 14 Mar 2012 14:51:01 -0700
Message-ID: 
 <CAC3rAA1VEDxW0GHCQTUhv_AfOpqHR=7cffopRXyXJOdVJK2yig@mail.gmail.com>
Subject: Re: [configuration] How to get threadsafe subset() config in a
 threadsafe way?
From: Tim Bessie <tbessie@meez.com>
To: Commons Users List <user@commons.apache.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Excuse me, mistake on the stacktrace commented "NullPointerException
on read of a configuration property" - that was a property setting,
not getting; if we find a non-existent property, we add it with a
default value.

- Tim

On Wed, Mar 14, 2012 at 2:48 PM, Tim Bessie <tbessie@meez.com> wrote:
> Surely - here's a few - these are all using commons-config 1.7. =A0Don't
> know if 1.8 would fix these; since they seemed linked to concurrency,
> I doubt it, but perhaps you know?
>
> ---------------------------------------------------------------
>
> I attempted to avoid ConcurrentModificationExceptions by creating a
> new HashMap from an existing SubsetConfiguration which I'd converted
> to a ConfigurationMap. =A0Unfortunately, the HashMap creation iterates
> through the underlying Configuration objects, and generates this
> error:
>
> [00:00:37.609]java.util.ConcurrentModificationException
> [00:00:37.609] =A0at
> org.apache.commons.collections.map.AbstractLinkedMap$LinkIterator.nextEnt=
ry(AbstractLinkedMap.java:559)
> [00:00:37.609] =A0at
> org.apache.commons.collections.map.AbstractLinkedMap$KeySetIterator.next(=
AbstractLinkedMap.java:459)
> [00:00:37.609] =A0at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(Fil=
terIterator.java:183)
> [00:00:37.609] =A0at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIte=
rator.java:93)
> [00:00:37.609] =A0at
> org.apache.commons.configuration.CompositeConfiguration.getKeys(Composite=
Configuration.java:236)
> [00:00:37.609] =A0at
> org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfig=
uration.java:196)
> [00:00:37.609] =A0at
> org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$Config=
urationSetIterator.<init>(ConfigurationMap.java:158)
> [00:00:37.609] =A0at
> org.apache.commons.configuration.ConfigurationMap$ConfigurationSet$Config=
urationSetIterator.<init>(ConfigurationMap.java:151)
> [00:00:37.609] =A0at
> org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.iterat=
or(ConfigurationMap.java:202)
> [00:00:37.609] =A0at java.util.HashMap.putAllForCreate(HashMap.java:536)
> [00:00:37.609] =A0at java.util.HashMap.<init>(HashMap.java:269)
>
> ---------------------------------------------------------------
>
> NullPointerException on read of a configuration property. =A0Digging
> into it, it *looks* like there's a null key value in a Map, but we
> don't put any null key values into our configuration. =A0Could be
> another error caused this to happen, or some concurrent modification
> elsewhere:
>
> java.lang.NullPointerException
> =A0 =A0 =A0 =A0at org.apache.commons.collections.map.AbstractLinkedMap.re=
moveEntry(AbstractLinkedMap.java:292)
> =A0 =A0 =A0 =A0at org.apache.commons.collections.map.AbstractHashedMap.re=
moveMapping(AbstractHashedMap.java:542)
> =A0 =A0 =A0 =A0at org.apache.commons.collections.map.AbstractHashedMap.re=
move(AbstractHashedMap.java:324)
> =A0 =A0 =A0 =A0at org.apache.commons.configuration.BaseConfiguration.clea=
rPropertyDirect(BaseConfiguration.java:133)
> =A0 =A0 =A0 =A0at org.apache.commons.configuration.AbstractConfiguration.=
clearProperty(AbstractConfiguration.java:503)
> =A0 =A0 =A0 =A0at org.apache.commons.configuration.CompositeConfiguration=
.clearPropertyDirect(CompositeConfiguration.java:269)
> =A0 =A0 =A0 =A0at org.apache.commons.configuration.AbstractConfiguration.=
clearProperty(AbstractConfiguration.java:503)
> =A0 =A0 =A0 =A0at org.apache.commons.configuration.AbstractConfiguration.=
setProperty(AbstractConfiguration.java:483)
> =A0 =A0 =A0 =A0at com.dm.avatar.configuration.Configuration.getProperty(C=
onfiguration.java:880)
> =A0 =A0 =A0 =A0at com.dm.avatar.configuration.Configuration.getProperty(C=
onfiguration.java:835)
> =A0 =A0 =A0 =A0at com.dm.avatar.configuration.Configuration.getProperty(C=
onfiguration.java:830)
> =A0 =A0 =A0 =A0at com.dm.web.util.CommonService.getHostString(CommonServi=
ce.java:795)
>
> -------------------------------------------
>
> Another NullPointerException case when wrapping a SubsetConfiguration
> in a ConfigurationMap with a new HashMap:
>
> [08:05:18.787]java.lang.NullPointerException
> [08:05:18.787] =A0at
> org.apache.commons.configuration.AbstractConfiguration$3.evaluate(Abstrac=
tConfiguration.java:577)
> [08:05:18.787] =A0at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(Fil=
terIterator.java:184)
> [08:05:18.787] =A0at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIte=
rator.java:93)
> [08:05:18.787] =A0at
> org.apache.commons.configuration.CompositeConfiguration.getKeys(Composite=
Configuration.java:236)
> [08:05:18.787] =A0at
> org.apache.commons.configuration.SubsetConfiguration.getKeys(SubsetConfig=
uration.java:196)
> [08:05:18.787] =A0at
> org.apache.commons.configuration.ConfigurationMap$ConfigurationSet.size(C=
onfigurationMap.java:189)
> [08:05:18.787] =A0at java.util.AbstractMap.size(AbstractMap.java:67)
> [08:05:18.787] =A0at java.util.HashMap.<init>(HashMap.java:267)
> [08:05:18.787] =A0at
> com.dm.avatar.configuration.Configuration.getPropertiesAsMap(Configuratio=
n.java:607)
>
>
>
>
> On Wed, Mar 14, 2012 at 1:24 PM, Oliver Heger
> <oliver.heger@oliver-heger.de> wrote:
>>
>> Am 13.03.2012 19:05, schrieb Tim Bessie:
>>
>>> That's a possibility which I'll look into.
>>>
>>> In the meantime, after initializing our main Configuration wrapper obje=
ct
>>> with all of our static configuration data, I copy slices of it to stati=
c
>>> maps that I know will never change, and provide convenience accessors t=
o
>>> those maps' data. =A0Since the calls that access the data placed in the=
se new
>>> maps were the only ones asking for slices of config data, this eliminat=
es
>>> our problem. =A0I was just hoping for a more elegant solution. =A0But a=
t least
>>> this way there's no locks necessary.
>>>
>>> I'll take a look at your suggestions, however - thanks much!
>>>
>>> A nice feature in the future could be something like
>>> ConfigurationUtils.getSynchronizedInstance(Configuration), for example.
>>> =A0I'm sure it's been discussed before. :-)
>>>
>>> - Tim
>>
>>
>> You are right, support for concurrent access to Configuration objects ca=
n certainly be improved.
>>
>> Can you provide a stack trace of such a ConcurrentModificationException =
you receive occasionally?
>>
>> Thanks
>> Oliver
>>
>>
>>>
>>> On Tue, Mar 13, 2012 at 1:43 AM, Luc Maisonobe<Luc.Maisonobe@free.fr>wr=
ote:
>>>
>>>> Le 13/03/2012 07:34, Tim Bessie a =E9crit :
>>>>>
>>>>> Hi all...
>>>>
>>>>
>>>> Hi Tim,
>>>>
>>>>>
>>>>> So we're keeping some config information CompositeConfiguration objec=
t,
>>>>
>>>> and
>>>>>
>>>>> we need to get subsets of this configuration data.
>>>>>
>>>>> When I call .subset(...), and then do some checks on the subset
>>>>
>>>> (isEmpty(),
>>>>>
>>>>> etc.), I sometimes get ConcurrentModificationExceptions. =A0I'm not s=
ure
>>>>> what's modifying the underlying configuration, although we do have
>>>>> occasional setting of configuration values throughout the running of =
our
>>>>> app.
>>>>>
>>>>> What would be *ideal* would be to:
>>>>>
>>>>> 1. NOT have to synchronize every access to the configuration object,
>>>>
>>>> since
>>>>>
>>>>> we have a high-volume application
>>>>> 2. To allow read and write operations to happen to the configuration
>>>>
>>>> object
>>>>>
>>>>> without worrying about ConcurrentModificationExceptions
>>>>> 3. To be able to call .subset(...) on the configuration object and 1)=
 not
>>>>> risk a ConcurrentModificationException during this operation, and 2) =
get
>>>>
>>>> a
>>>>>
>>>>> COPY of the subset back, so that further operations on the subset don=
't
>>>>> risk ConcurrentModificationExceptions
>>>>>
>>>>> Does anyone know of a way to do this? =A0Or is the ONLY way to guaran=
tee
>>>>
>>>> lack
>>>>>
>>>>> of CMEs to synchronize EVERY access (reads, writes, subsets, iteratio=
ns,
>>>>> etc.)?
>>>>
>>>>
>>>> Perhaps you could try java.util.concurrent.locks.ReadWriteLock ? You
>>>> would have to put the protection by yourself so it may require lots of
>>>> code wrapping. One of the good thins is that it allows concurrent read
>>>> (but when a write occurs, only one thread can write and reads are bloc=
ked).
>>>>
>>>> Luc
>>>>
>>>>>
>>>>> If this is the case, how have others dealt with situations like ours,
>>>>
>>>> where
>>>>>
>>>>> you do mostly just reads on a Configuration object, but very occasion=
al
>>>>> writes, and need to take subsets, and need to avoid exceptions while
>>>>
>>>> doing
>>>>>
>>>>> this? =A0With a high volume app, synchronizing every access would slo=
w
>>>>
>>>> things
>>>>>
>>>>> waaaaaay down, thus my question.
>>>>>
>>>>> - Tim
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: user-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>
>
>
> --
>
> Tim Bessie
> Meez, Inc.
> tbessie@meez.com


--=20

Tim Bessie
Meez, Inc.
tbessie@meez.com

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org