openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ezzio <dez...@apache.org>
Subject Re: Failures in TeamCity Derby build and svn commit: r616972
Date Wed, 06 Feb 2008 01:01:53 GMT
Hi Patrick,

The build was fragile because a couple of test cases were fragile (one 
did not clean out data before checking whether only the data it added 
was present, and the other did not ensure that time passed between 
updates when using a timestamp version check), and because the testing 
framework itself was fragile in that it attempted to delete data using a 
new entity manager when transactions might remain open (due to test 
failures), resulting in deadlock in the database.  I've fixed these 
issues.

Since there is no present resolution of the question whether to 
serialize or not the EMF, I have put in my change once again together 
with the updated test case.  I expect that TeamCity will now be happy 
due to the aforementioned fixes of the testing framework, etc.

Of course, we can always revisit the issue of whether to allow the EMF 
to be serialized, but since it currently is serializable, I put in the 
fix to actually allow it to serialize successfully.

David

Patrick Linskey wrote:
> Yes, I agree that if we make serialization not work, we should make
> the BF not implement Serializable. That is my preferred approach: even
> if Kodo needs to serialize a PMF for JNDI purposes or something, we
> could do that in the PMF impl, instead of in the BF.
> 
> IMO, the build is fragile because it runs multiple unrelated tests in
> the same VM with no strong isolation between tests. This leads to
> unexpected side effects sometimes. This can be frustrating, but the
> benefits are considerable: faster test run time, plus we get to find
> out about weird unintended side effects.
> 
> -Patrick
> 
> On 2/1/08, David Ezzio <dezzio@apache.org> wrote:
>> Hi,
>>
>> As to testing that the EMF objects are useful after serialization, this
>> is a reasonable expectation of any object that is an instance of the
>> Serializable interface.
>>
>> If it turns out that we don't want to support serializing useful EMF
>> objects, then we should take care to remove the Serializable interface
>> from their implementation.
>>
>> As I understand it, it is this latter issue that is currently under
>> discussion.
>>
>> An important, but only tangentially related, issue is why the build is
>> so fragile.
>>
>> David
>>
>> Pinaki Poddar wrote:
>>> Hi,
>>> Serializablity requirement on EMF is arising out of necessity rather
>>> than by JPA specifcation.
>>> JNDI, too, does not mandate bound objects to be Serializable.
>>> But JNDI providers often place that restriction.
>>> Non-serializable objects can be bound to JNDI by implementing
>>> javax.naming.Referenceable.
>>> Kodo's JDO product derivation based on OpenJPA is following this
>>> Referenceable route.
>>>
>>> But I do not see this in openjpa EntityManagerFactory implementation.
>>>
>>>
>>> -----Original Message-----
>>> From: Kevin Sutter [mailto:kwsutter@gmail.com]
>>> Sent: Friday, February 01, 2008 11:11 AM
>>> To: dev@openjpa.apache.org
>>> Subject: Re: Failures in TeamCity Derby build and svn commit: r616972
>>>
>>> On Feb 1, 2008 10:17 AM, Pinaki Poddar <ppoddar@bea.com> wrote:
>>>
>>>> Factories needs to be Serializable as they are often placed in JNDI.
>>>
>>> The JNDI lookup needs to work for the Factory definition, but it doesn't
>>> require that the actual Factory be present in JNDI.  And, since the JNDI
>>> lookup is meant for the EE (Container) environment, I would guess that
>>> most implementations provide a wrapper object in JNDI for the actual
>>> Factory.
>>>
>>> Granted, this wrapper approach is not a requirement.  But, I can't find
>>> the requirement in the spec that the EMF has to be Serializable
>>> either...  :-)
>>>
>>> Kevin
>>>
>>>
>>>> -----Original Message-----
>>>> From: Patrick Linskey [mailto:plinskey@gmail.com]
>>>> Sent: Thursday, January 31, 2008 7:57 PM
>>>> To: dev@openjpa.apache.org
>>>> Subject: Re: Failures in TeamCity Derby build and svn commit: r616972
>>>>
>>>> To be honest, I don't really understand why a BrokerFactory needs to
>>>> be serializable, so I think probably it'd be good to make sure that
>>>> there is good reason for that before going too deep into this
>>>> debugging exercise.
>>>>
>>>> -Patrick
>>>>
>>>> On Jan 31, 2008 5:52 PM, Patrick Linskey <plinskey@gmail.com> wrote:
>>>>>> If the hangup is intolerable, let me know, and I'll back out my
>>>>>> change tomorrow.
>>>>> Personally, I'd rather have the TeamCity build be clean while you
>>>>> investigate. As you noted, the error-reporting from surefire is
>>>>> frustratingly inaccurate, and this really erodes the benefit of
>>>>> continuous integration.
>>>>>
>>>>>> It would be handy if I knew a few things.  How to pass in VM debug
>>>>>> args to Maven for its test runs, where to set the OpenJPA logging
>>>>>> when Maven runs the tests, and how to pass in the appropriate
>>>>>> parameters when running the test case outside of Maven.  Any
>>>>>> insights on these questions will be appreciated.
>>>>> I usually just add log configuration to the test cases themselves,
>>>>> in the setUp(...) method:
>>>>>
>>>>>     setUp(Class.class, Class2.class, ..., "openjpa.Log",
>>>>> "SQL=TRACE");
>>>>>
>>>>> I believe that it is also possible to set this at the command line
>>>>> via
>>>>> -Dopenjpa.Log=SQL=TRACE types of clauses.
>>>>>
>>>>> -Patrick
>>>>>
>>>>>
>>>>> On Jan 31, 2008 4:08 PM, David Ezzio <dezzio@apache.org> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Progress has been slow on the test failures apparently introduced
>>>>>> by
>>>>>> 616972.  I have made progress in reproducing the results.
>>>>>>
>>>>>> The miscounting in TeamCity is gross.  Although it lists 33
>>>>>> failing tests complete with a credible stack trace for each, in
>>>>>> fact, there is only one failing test.
>>>>>>
>>>>>> Meanwhile, I believe now, that change 616658 is no longer a
>>> suspect.
>>>>>> I get very erratic results here, but they seem to vary more by
>>>>>> whether I run "mvn test" after "mvn clean" or "mvn install" after
>>>>>> "mvn clean".  By doing just what TeamCity does, "mvn clean compile
>>>>>> test", I can finally reproduce the results locally that TeamCity
>>>>>> reports in its Compile-Derby build for this change (395, I think).
>>>>>> In spite of this, if I run "mvn
>>>>>> -Dtest=TestCaseInsensitiveKeywordsInJPQL clean compile test" the
>>>> test passes.
>>>>>> It would be handy if I knew a few things.  How to pass in VM debug
>>>>>> args to Maven for its test runs, where to set the OpenJPA logging
>>>>>> when Maven runs the tests, and how to pass in the appropriate
>>>>>> parameters when running the test case outside of Maven.  Any
>>>>>> insights on these questions will be appreciated.
>>>>>>
>>>>>> But at least at this point, I can reproduce locally what TeamCity
>>>>>> reports, and although it happened only once out of one try, I am
>>>>>> optimistic that it is reproducible.  I will continue working this
>>>>>> tomorrow.
>>>>>>
>>>>>> If the hangup is intolerable, let me know, and I'll back out my
>>>>>> change tomorrow.
>>>>>>
>>>>>> David Ezzio
>>>>>>
>>>>>>
>>>>>> David Ezzio wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am convinced that change 616658 (small change to
>>>>>>> DBDictionary.java) is the source of the two intermittent
>>>>>>> failures
>>>> that I am seeing locally.
>>>>>>> One of those intermittent failures is the failure in
>>>>>>> TestCaseInsensitiveKeywordsInJPQL.testCaseInsensitiveBooleans
>>>>>>> which is one of the 33 related failures currently seen in
>>>>>>> TeamCity
>>>> Derby builds.
>>>>>>> The other intermittent failure is
>>>>>>> TestVersion.testVersionTimestamp
>>>>>>> which I reported earlier as occurring locally with change
>>> 616658.
>>>>>>> When I revert change 616658 locally, both of these issues go
>>>>>>> away even with the changes that I submitted to
>>>>>>> AbstractBrokerFactory in
>>>>>>> change 616972.
>>>>>>>
>>>>>>> I have a question: is it possible to run a build in TeamCity
>>>>>>> without submitting the change (or conditionally submitting the
>>>> change) to SVN?
>>>>>>> I'd like to find out if reverting 616658 will fix the issue.
 If
>>>>>>> so, then I can try to figure out why it's a problem.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> David Ezzio wrote:
>>>>>>>> No, but I'm taking a look since my changes appear to be the
>>>> cause.
>>>>>>>> Patrick Linskey wrote:
>>>>>>>>> 5. The tests are now failing in the automated build;
any idea
>>>>>>>>> why this might be?
>>>>>>>>>
>>>>>>>>> -Patrick
>>>>>>>>>
>>>>>>>>> On Jan 30, 2008 6:11 PM, Patrick Linskey <plinskey@gmail.com>
>>>> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> A few questions:
>>>>>>>>>>
>>>>>>>>>> 1. Were you seeing this cause problems somewhere?
I.e., when
>>>>>>>>>> do
>>>>>>>>>> we serialize BrokerFactory instances?
>>>>>>>>>>
>>>>>>>>>> 2. Why not just move the initialization code to the
>>>>>>>>>> declaration
>>>>>>>>>> line (instead of the constructor)?
>>>>>>>>>>
>>>>>>>>>> 3. It looks like there are other transactional structures
>>>>>>>>>> that might not be properly handled in serialization
/
>>>>>>>>>> deserialization (lifecycle listeners, for example).
Should we
>>>> be making more changes there?
>>>>>>>>>> -Patrick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jan 30, 2008 4:59 PM,  <dezzio@apache.org>
wrote:
>>>>>>>>>>> Author: dezzio
>>>>>>>>>>> Date: Wed Jan 30 16:59:02 2008 New Revision:
616972
>>>>>>>>>>>
>>>>>>>>>>> URL: http://svn.apache.org/viewvc?rev=616972&view=rev
>>>>>>>>>>> Log:
>>>>>>>>>>> Allow EntityManagerFactory objects to be serialized
and
>>>>>>>>>>> deserialized successfully.
>>>>>>>>>>>
>>>>>>>>>>> Added:
>>>>>>>>>>>
>>>>>>>>>>> openjpa/trunk/openjpa-persistence-jdbc/src/test/java/org/apa
>>>>>>>>>>> ch e/openjpa/persistence/simple/TestSerializedFactory.java
>>>>>>>>>>> (with props)
>>>>>>>>>>> Modified:
>>>>>>>>>>>
>>>>>>>>>>> openjpa/trunk/openjpa-kernel/src/main/java/org/apache/openjp
>>>>>>>>>>> a/ kernel/AbstractBrokerFactory.java
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Modified:
>>>>>>>>>>> openjpa/trunk/openjpa-kernel/src/main/java/org/apache/openjp
>>>>>>>>>>> a/ kernel/AbstractBrokerFactory.java
>>>>>>>>>>>
>>>>>>>>>>> URL:
>>>>>>>>>>> http://svn.apache.org/viewvc/openjpa/trunk/openjpa-kernel/sr
>>>>>>>>>>> c/
>>>>>>>>>>> main/java/org/apache/openjpa/kernel/AbstractBrokerFactory.ja
>>>>>>>>>>> va ?rev=616972&r1=616971&r2=616972&view=diff
>>>>>>>>>>>
>>>>>>>>>>> ============================================================
>>>>>>>>>>> ==
>>>>>>>>>>> ================
>>>>>>>>>>>
>>>>>>>>>>> ---
>>>>>>>>>>> openjpa/trunk/openjpa-kernel/src/main/java/org/apache/openjp
>>>>>>>>>>> a/ kernel/AbstractBrokerFactory.java
>>>>>>>>>>> (original)
>>>>>>>>>>> +++
>>>>>>>>>>> openjpa/trunk/openjpa-kernel/src/main/java/org/apache/openjp
>>>>>>>>>>> a/ kernel/AbstractBrokerFactory.java Wed Jan
30 16:59:02
>>>>>>>>>>> 2008 @@ -147,8 +147,7 @@
>>>>>>>>>>>       */
>>>>>>>>>>>      protected AbstractBrokerFactory(OpenJPAConfiguration
>>>> config) {
>>>>>>>>>>>          _conf = config;
>>>>>>>>>>> -        _pcClassLoaders = new ConcurrentReferenceHashSet(
>>>>>>>>>>> -            ConcurrentReferenceHashSet.WEAK);
>>>>>>>>>>> +        getPcClassLoaders();
>>>>>>>>>>>      }
>>>>>>>>>>>
>>>>>>>>>>>      /**
>>>>>>>>>>> @@ -287,13 +286,13 @@
>>>>>>>>>>>                      if (needsSub(cls))
>>>>>>>>>>>                          toRedefine.add(cls);
>>>>>>>>>>>                  }
>>>>>>>>>>> -                _pcClassLoaders.add(loader);
>>>>>>>>>>> +                getPcClassLoaders().add(loader);
>>>>>>>>>>>                  _pcClassNames = c;
>>>>>>>>>>>              }
>>>>>>>>>>>              _persistentTypesLoaded = true;
>>>>>>>>>>>          } else {
>>>>>>>>>>>              // reload with this loader
>>>>>>>>>>> -            if (_pcClassLoaders.add(loader))
{
>>>>>>>>>>> +            if (getPcClassLoaders().add(loader))
{
>>>>>>>>>>>                  for (Iterator itr =
>>>>>>>>>>> _pcClassNames.iterator();
>>>>>>>>>>> itr.hasNext();) {
>>>>>>>>>>>                      try {
>>>>>>>>>>>                          Class cls = @@ -818,4
+817,15 @@
>>>>>>>>>>>              _transactional.remove (_trans);
>>>>>>>>>>>                 }
>>>>>>>>>>>         }
>>>>>>>>>>> +
>>>>>>>>>>> +   /**
>>>>>>>>>>> +    * Method insures that deserialized EMF has
this
>>>>>>>>>>> + reference
>>>>>>>>>>> re-instantiated
>>>>>>>>>>> +    */
>>>>>>>>>>> +   private Collection getPcClassLoaders() {
>>>>>>>>>>> +      if (_pcClassLoaders == null)
>>>>>>>>>>> +        _pcClassLoaders = new ConcurrentReferenceHashSet(
>>>>>>>>>>> +            ConcurrentReferenceHashSet.WEAK);
>>>>>>>>>>> +
>>>>>>>>>>> +      return _pcClassLoaders;
>>>>>>>>>>> +   }
>>>>>>>>>>>  }
>>>>>>>>>>>
>>>>>>>>>>> Added:
>>>>>>>>>>> openjpa/trunk/openjpa-persistence-jdbc/src/test/java/org/apa
>>>>>>>>>>> ch e/openjpa/persistence/simple/TestSerializedFactory.java
>>>>>>>>>>>
>>>>>>>>>>> URL:
>>>>>>>>>>> http://svn.apache.org/viewvc/openjpa/trunk/openjpa-persisten
>>>>>>>>>>> ce
>>>>>>>>>>> -jdbc/src/test/java/org/apache/openjpa/persistence/simple/Te
>>>>>>>>>>> st SerializedFactory.java?rev=616972&view=auto
>>>>>>>>>>>
>>>>>>>>>>> ============================================================
>>>>>>>>>>> ==
>>>>>>>>>>> ================
>>>>>>>>>>>
>>>>>>>>>>> ---
>>>>>>>>>>> openjpa/trunk/openjpa-persistence-jdbc/src/test/java/org/apa
>>>>>>>>>>> ch e/openjpa/persistence/simple/TestSerializedFactory.java
>>>>>>>>>>> (added)
>>>>>>>>>>> +++
>>>>>>>>>>> openjpa/trunk/openjpa-persistence-jdbc/src/test/java/org/apa
>>>>>>>>>>> ch e/openjpa/persistence/simple/TestSerializedFactory.java
>>>>>>>>>>> Wed Jan 30 16:59:02 2008
>>>>>>>>>>> @@ -0,0 +1,74 @@
>>>>>>>>>>> +/*
>>>>>>>>>>> + * Licensed to the Apache Software Foundation
(ASF) under
>>>>>>>>>>> +one
>>>>>>>>>>> + * or more contributor license agreements. 
See the NOTICE
>>>>>>>>>>> +file
>>>>>>>>>>> + * distributed with this work for additional
information
>>>>>>>>>>> + * regarding copyright ownership.  The ASF licenses
this
>>>>>>>>>>> +file
>>>>>>>>>>> + * to you under the Apache License, Version
2.0 (the
>>>>>>>>>>> + * "License"); you may not use this file except
in
>>>>>>>>>>> +compliance
>>>>>>>>>>> + * with the License.  You may obtain a copy
of the License
>>>>>>>>>>> +at
>>>>>>>>>>> + *
>>>>>>>>>>> + * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>>>> + *
>>>>>>>>>>> + * Unless required by applicable law or agreed
to in
>>>>>>>>>>> +writing,
>>>>>>>>>>> + * software distributed under the License is
distributed on
>>>>>>>>>>> +an
>>>>>>>>>>> + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
OF ANY
>>>>>>>>>>> + * KIND, either express or implied.  See the
License for
>>>>>>>>>>> +the
>>>>>>>>>>> + * specific language governing permissions and
limitations
>>>>>>>>>>> + * under the License.
>>>>>>>>>>> + */
>>>>>>>>>>> +package org.apache.openjpa.persistence.simple;
>>>>>>>>>>> +
>>>>>>>>>>> +import java.io.*;
>>>>>>>>>>> +
>>>>>>>>>>> +import javax.persistence.EntityManager; import
>>>>>>>>>>> +javax.persistence.EntityManagerFactory;
>>>>>>>>>>> +import javax.persistence.EntityTransaction;
>>>>>>>>>>> +
>>>>>>>>>>> +import junit.textui.TestRunner; import
>>>>>>>>>>> +org.apache.openjpa.persistence.OpenJPAEntityManager;
>>>>>>>>>>> +import
>>>>>>>>>>> +org.apache.openjpa.persistence.test.SingleEMFTestCase;
>>>>>>>>>>> +
>>>>>>>>>>> +/**
>>>>>>>>>>> + * Tests that a EntityManagerFactory can be
used after
>>>> serialization.
>>>>>>>>>>> + *
>>>>>>>>>>> + * @author David Ezzio
>>>>>>>>>>> + */
>>>>>>>>>>> +public class TestSerializedFactory
>>>>>>>>>>> +    extends SingleEMFTestCase {
>>>>>>>>>>> +
>>>>>>>>>>> +    public void setUp() {
>>>>>>>>>>> +        setUp(AllFieldTypes.class);
>>>>>>>>>>> +    }
>>>>>>>>>>> +
>>>>>>>>>>> +    public void testSerializedEntityManagerFactory()
throws
>>>>>>>>>>> Exception {
>>>>>>>>>>> +        // serialize and deserialize the entity
manager
>>>> factory
>>>>>>>>>>> +        ByteArrayOutputStream baos = new
>>>> ByteArrayOutputStream();
>>>>>>>>>>> +        ObjectOutputStream oos = new
>>>> ObjectOutputStream(baos);
>>>>>>>>>>> +        oos.writeObject(emf);
>>>>>>>>>>> +        EntityManagerFactory emf2 =
>>>>>>>>>>> +            (EntityManagerFactory) new ObjectInputStream(
>>>>>>>>>>> +            new
>>>>>>>>>>> ByteArrayInputStream(baos.toByteArray())).readObject();
>>>>>>>>>>> +
>>>>>>>>>>> +        // use the deserialized entity manager
factory
>>>>>>>>>>> +        assertTrue("The deserialized entity
manager factory
>>>>>>>>>>> + is not
>>>>>>>>>>> open",
>>>>>>>>>>> +            emf2.isOpen());
>>>>>>>>>>> +        EntityManager em = emf2.createEntityManager();
>>>>>>>>>>> +        assertTrue("The newly created entity
manager is not
>>>>>>>>>>> + open",
>>>>>>>>>>> em.isOpen());
>>>>>>>>>>> +
>>>>>>>>>>> +        // exercise the entity manager produced
from the
>>>>>>>>>>> deserialized EMF
>>>>>>>>>>> +        em.getTransaction().begin();
>>>>>>>>>>> +        em.persist(new AllFieldTypes());
>>>>>>>>>>> +        em.getTransaction().commit();
>>>>>>>>>>> +
>>>>>>>>>>> +        // close the extra resources
>>>>>>>>>>> +        em.close();
>>>>>>>>>>> +        assertFalse("The entity manager is not
closed",
>>>> em.isOpen());
>>>>>>>>>>> +        emf2.close();
>>>>>>>>>>> +        assertFalse("The entity manager factory
is not
>>>>>>>>>>> + closed",
>>>>>>>>>>> emf2.isOpen());
>>>>>>>>>>> +    }
>>>>>>>>>>> +
>>>>>>>>>>> +    public static void main(String[] args) {
>>>>>>>>>>> +        TestRunner.run(TestSerializedFactory.class);
>>>>>>>>>>> +    }
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>>
>>>>>>>>>>> Propchange:
>>>>>>>>>>> openjpa/trunk/openjpa-persistence-jdbc/src/test/java/org/apa
>>>>>>>>>>> ch e/openjpa/persistence/simple/TestSerializedFactory.java
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> --
>>>>>>>>>>> ----------------
>>>>>>>>>>>
>>>>>>>>>>>     svn:eol-style = native
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Patrick Linskey
>>>>>>>>>> 202 669 5907
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>> Patrick Linskey
>>>>> 202 669 5907
>>>>>
>>>>
>>>> --
>>>> Patrick Linskey
>>>> 202 669 5907
>>>>
>>>> Notice:  This email message, together with any attachments, may
>>>> contain information  of  BEA Systems,  Inc.,  its subsidiaries  and
>>>> affiliated entities,  that may be confidential,  proprietary,
>>>> copyrighted  and/or legally privileged, and is intended solely for the
>>>> use of the individual or entity named in this message. If you are not
>>>> the intended recipient, and have received this message in error,
>>>> please immediately return this by email and then delete it.
>>>>
>>> Notice:  This email message, together with any attachments, may contain
>> information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated
>> entities,  that may be confidential,  proprietary,  copyrighted  and/or
>> legally privileged, and is intended solely for the use of the individual or
>> entity named in this message. If you are not the intended recipient, and
>> have received this message in error, please immediately return this by email
>> and then delete it.
> 
> 

Mime
View raw message