ofbiz-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shi Jinghai <huaru...@hotmail.com>
Subject Re: Why A Framework Rewrite Is Necessary
Date Thu, 15 Oct 2015 00:39:52 GMT
As Jacques mentioned, OFBiz is a bit special in Apache. Why special? Personally, I think OFBiz
is built for Sharan, for consultants. The other projects in Apache are for developers.

So my vote will follow Sharan's. :)


-----邮件原件-----
发件人: Adrian Crum [mailto:adrian.crum@sandglass-software.com] 
发送时间: 2015年10月15日 3:21
收件人: dev@ofbiz.apache.org
主题: Why A Framework Rewrite Is Necessary

I understand that Sharan brought up the framework rewrite subject at 
ApacheCon, and some attendees felt that the framework is fine and no 
action needs to be taken.

In this message, I will try to give a detailed explanation of why a 
framework rewrite is necessary. I don't plan to take any further action 
on this subject, because I've brought it up before without success, and 
I'm tired of discussing it. It is my hope that the light bulb will click 
on in someone's head and they will take action.

My Background
-------------

I became a member of the OFBiz community in 2004. I immediately started 
making contributions to the project by supplying patches to the issue 
tracker. In 2007, I became a committer. Most of my initial work was on 
the UI and some work in the applications (mainly Asset Maintenance and 
Work Effort). I stayed away from touching the framework code because it 
was deep, dark, and scary.

Eventually, I started to understand how the framework code works and I 
made some minor modifications. As my understanding grew, I progressed to 
rewriting large swaths of framework code - making it thread-safe, fault 
tolerant, efficient, and easier to use.

I will list some of my contributions here, so everyone can have a clear 
understanding of my experience with the framework code:

     New Features

         User Preferences

         Visual Themes

         Custom UI Label XML File Format

         Temporal Expressions

         Data Type Conversion Framework

         Screen Widget Boundary Comments

         Metrics

     Integrations

         UEL

         iCalendar

         JSR 223

         WebDAV

         LDAP

     Refactorings/Improvements

         FlexibleStringExpander

         FlexibleMapExpander

         FOP Integration

         FreeMarkerWorker

         Date-Time Handling

         Mini-language

         Job Scheduler

In addition, I have performed innumerable framework bug fixes.

So, the contents of this message come from years of experience mucking 
about in the framework code.

Okay, let's get started...

Initial Problem Statement
-------------------------

In 2009, David Jones started a framework rewrite in a branch:

https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716

At the time, there was some agreement that a rewrite was necessary, but 
there was disagreement as to how the rewrite should be incorporated into 
the project:

https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%3C455601.62605.qm@web63102.mail.re1.yahoo.com%3E

There were concerns that a rewrite would break backward compatibility. 
Work on the rewrite branch stopped. Eventually, Jacopo suggested the 
community be more accepting of backward-incompatible changes:

https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%3cD24F129D-4F9F-444E-84AF-ACA46F49990C@hotwaxmedia.com%3e

Despite an effort to convince David to proceed with the framework 
rewrite, he ended up doing it in a separate project:

http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%3C07565C88-4023-4D24-93A3-A4906E86F939@me.com%3E

This page describes differences between OFBiz and Moqui, and within it 
you can extract information on the problems David was trying to solve:

http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/

There was an email he sent out on the OFBiz dev list where he listed the 
problems he saw in the framework, but I can't find it. The rest of this 
message will include the issues he mentioned (the ones I remember). I 
was in agreement with him at the time, and I still agree that a 
framework rewrite is necessary.

The Problems
------------

Code is scattered everywhere - due to an initial effort to make the 
framework modular. This causes serious problems. The mere fact that 
components like entityext and securityext EXIST makes it clear that 
there are problems - those components should not be there. Also, we run 
into the recurring problem of circular dependencies (component A will 
not build unless component B is built, and component B will not build 
unless component A is built).

Bad separation of concerns. There are far too many examples of classes 
that try to be everything to everyone. This makes debugging difficult, 
and it makes maintenance/improvements a nightmare. [Using an analogy, 
consider an automobile design where a spark plug is not separate from a 
transmission. Instead, the automobile uses a spark-plug-transmission. So 
when the engine is running rough because the spark plug is bad, you have 
to replace the spark plug AND the transmission.] A good framework 
example can be found in my rewrite of the mini-language code. 
Originally, the models AND the script execution context both contained 
script behaviors - making debugging/improvements difficult. I changed it 
so only the models contain script behavior and the script execution 
context contains only the script execution state.

Lack of good OO design. There are many places where a bit of framework 
functionality is contained in a single method that is hundreds or 
thousands of lines long. There is a term for that: Brittle Code. Code 
isn't reused. Instead, it is copy-and-pasted all over - so when a 
problem is found in the C&P code, it has to be fixed in many places 
instead of one.

Fail-slow design. There are a lot of places in low-level code where an 
error condition is encountered, but instead of throwing an exception, 
the error is ignored and maybe it is logged, or the code tries to 
"guess" at a solution and then provide an arbitrary default behavior. 
I've seen many developers struggle with debugging a problem because they 
didn't look at the logs, or because the error was not logged and there 
is no way of knowing what caused it. They end up spending hours 
single-stepping through code until it reaches the error.

Out-of-date code. A good example is the use of Javolution. That library 
was beneficial in the Java 1.4 days, but it is not necessary today 
because of improved garbage collection. Another good example is DCL 
code. DCL was used extensively in OFBiz, but it is clearly documented to 
be an unreliable design (I can get it to fail 90% of the time). Some DCL 
code has been replaced, but a lot of it is still there.

Portions of the API are overly complicated. Some methods require a 
collection of user-specified artifacts/arguments, which makes client 
code complicated and verbose. (David solved that problem with his 
Execution Context.) Portions of the API are cluttered with unnecessary 
"convenience methods" - making the API harder to learn and memorize. In 
some places, a domain-specific API is spread across instance methods and 
static methods and across different classes - making the API hard to 
understand and use. Yes, there can be good designs that require 
something like that, but in the OFBiz framework, it exists because of a 
bad design, not a good one.

Use of thread-local variables. This makes multi-threaded design 
impossible. The J2EE specification and the Servlet API require one 
thread per request (and most J2EE libraries depend on that behavior), so 
the current design makes sense from a J2EE perspective, but what if I 
don't want to run the framework in a J2EE container? Which leads to the 
next problem...

Dependence on J2EE designs/APIs/libraries. There are developers in the 
Java community (myself included) who are beginning to question if J2EE 
is really necessary to run web applications. The folks at Atomikos are a 
good example. OFBiz does not use EJBs, so tying the framework to J2EE 
does not make sense. It would be better if the framework was designed to 
run outside a J2EE container, and then have container integration as an 
option.

Configuration files are scattered everywhere. Anyone who has deployed 
OFBiz in a production environment will agree this is a problem. Try 
changing the HTTP/HTTPS and port settings - it is a nightmare. Some 
configuration settings are in nonsensical places.

An abysmal lack of unit testing. I don't have an exact figure for code 
coverage, but my gut feeling is coverage is less than 10%. Basically, we 
all have our fingers crossed - hoping that the framework code works as 
expected. This was made painfully obvious a while back when I was 
looking at some entity caching code and thought to myself "this code 
can't work." So I wrote some entity cache unit tests and confirmed that 
the entity cache had serious problems. Think about that - years passed 
with no entity cache unit tests and consequently we had no idea it 
wasn't working.

Fix Versus Rewrite
------------------

Jira issues could be created for these problems and teams of developers 
could work to fix them.

Or, we could create a branch and start over from scratch. This time 
around, there should be less push-back from people concerned about 
backwards compatibility. A rewrite offers the advantage of reconsidering 
everything - like API design, general problem solving, and new features.

I created a Wiki page for a framework design:

https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision

but there hasn't been much interest in it. If the community decides to 
go ahead with a rewrite, then please feel free to use the Wiki pages as 
a guide.

Sandglass Foundation
--------------------

Like David, I came to the conclusion that a framework rewrite would be 
easier outside the OFBiz community. So, I created my own library called 
Foundation:

http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf

(PDF)

and I only mention it here to stress how wonderful it can be to start 
with a clean slate and design an API that is concise yet powerful. 
(Please do not discuss Foundation here, contact me privately if you want 
more information.)

Some examples of what can be done with a rewrite:

     A single configuration file
     Use ANSI/ISO SQL SELECT statement strings instead of constructing 
complicated Java structures
     Simultaneous asynchronous queries
     Relational integrity across multiple datasources
     Multi-table SELECT across multiple datasources
     Automatic and transparent row version control
     Automatic and transparent multi-language datasource support
     Abstract entities (similar to SQL user types)
     Service engine throttling (protects against server over-utilization)
     Simplified security (authorization) 
(https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign)
     Pure interface-based API - so developers are free to modify 
framework behavior by using decorators
     Thorough unit tests

Benefits of a rewrite:

     Reduced resource requirements (lower hosting fees)
     Reduce application development time - due to a simplified API
     Easier framework code maintenance
     Better reliability

Conclusion
----------

Like I said at the start, this is all I will say about the subject. I'm 
done trying to convince everyone. I hope someone agrees with me and they 
are able to build support for the idea.

-- 
Adrian Crum
Sandglass Software
www.sandglass-software.com


Mime
View raw message