Mailing-List: contact dev-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geronimo.apache.org
Received-SPF: pass (asf.osuosl.org: domain of david.blevins@visi.com
 designates 208.42.156.9 as permitted sender)
Mime-Version: 1.0 (Apple Message framework v734)
In-Reply-To: <4332AC9D.20601@gmail.com>
References: <4332AC9D.20601@gmail.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <CA7C8BF1-AC03-416A-A90D-583E699FCEF0@visi.com>
Content-Transfer-Encoding: 7bit
From: David Blevins <david.blevins@visi.com>
Subject: Re: Java serizalization compatibility issues
Date: Thu, 22 Sep 2005 13:34:56 -0700
To: dev@geronimo.apache.org

On Sep 22, 2005, at 6:07 AM, Sachin Patel wrote:

> So if you are not aware, I'm pulling in and packaging several jars  
> from the lib and lib/endorsed directory into one of the eclipse  
> plug-in, so the classes can be used and referenced by the rest of  
> the eclipse plugins.  This is because eclipse can not reference  
> classes or jars at runtime that are not packaged within a plug-in  
> and marked as visible in either the plugin.xml or manifest.
> A big problem resides as now the same jars I'm packaging must be  
> the same exact jars that reside in the target server I'm  
> deploying.  This causes a dependency on a particular server image.   
> If a user modifies classes I reference and rebuilds their server,  
> the plug-in is broken as during deployment I'll receive error  
> messages like the following...
>
> Caused by: java.io.InvalidClassException:  
> org.apache.geronimo.kernel.config.ConfigurationModuleType; local  
> class incompatible: stream classdesc serialVersionUID =  
> 6296527685792707191, local class serialVersionUID =  
> -4121586344416418391
>   So looking at that particular class, it looks like the  
> serialVersionUID is generated by Java compiler.  This is bad as now  
> jars/classes risk compatibility between every build.  We need a  
> solution for this.  The only other option I'm aware of is for these  
> serializable classes to hard code and explicity assign a value.  Of  
> course we must then assue that we manually maintain backward  
> compatibility to support the N-2 model for these classes.  This  
> problem will eventually have to be solved anyway when there is  
> multiple server support and across different versions.
> I think this is a must fix for 1.0 as without doing so we risk  
> product migration, mixed version interoperability, tooling,  
> possibly user applications, etc...
>

I agree with you.  We need a solution for this and it does affect  
compatibility even to the extent of breaking it on every build.

There was a one week battle on this in May that resulted in people  
saying "I dont' want to talk about it anymore."  I think the problem  
is an issue that can't be solved through serialization in an  
effective and reliable way.  Largely because of how easy it is to  
break serialized data and that broken serialized data is nearly  
unrecoverable.

Here is my perspective:

   With the proper amount of forethought and code, you can get a
   very limited amount of compatibility with serialization between
   class file versions, but it's unrealistic and unreliable at best.

   Unless you can convince everyone to do that, this entire thread
   is moot regardless of what you or I think, and our existing
   system will never serve as a reliable way to store configuration
   data across upgrades or patches.


Here are a few snippets of that argument, completely biased to my  
perspective:

----------------
On May 13, 2005, at 3:54 PM, David Blevins wrote:
> On Fri, May 13, 2005 at 03:42:58PM -0700, Jeremy Boynes wrote:
>>
>> Solving the compatibility problem will require the developers to pay
>> attention to versioning. We will need to require that at some  
>> point though.
>>
>
> Paying attention is an understatement.  In order to serialize data  
> to and from class definitions, those classes cannot change in any  
> significant way, ever.  No refactoring, repackaging, reorganizing,  
> no removing methods, no removing class variables.  This doesn't  
> just apply to public APIs, but even the internal classes of each  
> component.
>
> Considering that at deploy time we are building and serializing  
> Jetty web containers, Tomcat web containers, ActiveMQ queues/ 
> topics, OpenEJB ejb containers, Axis webservices, TranQL connectors  
> and more, is it reasonable to ask or expect these projects  
> basically not to change their code?
>
> Clearly, projects will clean and improve their code, we can't and  
> shouldn't try and stop them.  We want them to optimize and improve  
> things.  As this change is going to occur, people *will* have to  
> redeploy their apps as we support no other way to store  
> configurations.
>
> Serializing and deserializing server state for fast startup is a  
> cool feature, but not something we are in any position to guarantee  
> between releases of Axis, ActiveMQ, OpenEJB, Jetty, Tomcat, et. al.

----------------
On May 19, 2005, at 9:35 AM, David Blevins wrote:
> On Wed, May 18, 2005 at 09:04:07PM -0700, Jeremy Boynes wrote:
>
>> David Blevins wrote:
>>
>>>
>>> I know exactly how much room for error there is as well as the  
>>> amount
>>> forethought and code required to guarantee even a small amout of
>>> future compatibility.  Using SUIDs alone does not cut it. I'm not
>>> arguing it's impossible, I'm arguing that it's unrealistic and
>>> unreliable at best.
>>>
>>>
>>
>> I think you're looking at it backward - without a crystal ball you  
>> don't
>> know what future versions of the class will look like. It is the
>> responsbility of the evolved class to handle upgrading older data.
>>
>
> You won't know what order the fields were written to the stream, so  
> unless you had the forethought to write them yourself from the get- 
> go, you will be extremely limited on what you can do later.
>
> Even renaming a private variable would be impossible without  
> planning ahead.
>
>
>>> The major technical difference between xml data and object
>>> serialization data is that xml data can still be parsed even if the
>>> schema has changed. The same is not true of serialized object data.
>>>
>>
>> Why? You can parse a stream of bytes as easily as you can parse a  
>> stream
>> of characters. Whether you can use the result of the parse will  
>> depend
>> on the semantic content of the new infoset - you can create  
>> incompatible
>> XML schemas as well.
>>
>
> Simple, a well-formed XML document is always parseable weather or  
> not you choose to validate it against the schema.  Changing element  
> name, orders, or types can be dealt with later.  This is not the  
> case with object serialization where the class metadata is written  
> to the stream and you have no idea what it looks like.
>
> You cannot see, use, create, or predict the order of the class  
> metadata written to the stream.  I would hardly say it's any  
> comparison to XML Schema.
>
> As I keep saying, with the proper amount of forethought and code,  
> you can get a very limited amount of compatibility with  
> serialization between class file versions, but it's unrealistic and  
> unreliable at best.
>
> Unless you can convince everyone to do that, this entire thread is  
> moot regardless of what you or I think, and our existing system  
> will never serve as a reliable way to store configuration data  
> across upgrades or patches.
>
----------------