cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grzegorz Kossakowski <>
Subject Re: request encoding conundrum
Date Sun, 27 Jul 2008 17:51:50 GMT
Jeremy Quinn pisze:
> I still have all of the notes and the builds we did (thanks!).
> But I am still doing the work in 2.1, as (if I remember properly) we did 
> not manage to make a build that would edit live at the level of the 
> cforms block itself.
> Correct me if I am wrong, but it seems easier to setup 2.1 so that edits 
> made to the built-in resources of the block are immediately live without 
> re-building.

Jeremy, yep, you are wrong here. ;-)

Actually, time passed since our last meeting and I learned how to do things in damn easy way.
here goes the instructions:
1. Grab Eclipse from (I use 3.3 but newest 3.4 should work as well)
2. Install run jetty run plug-in for eclipse from:
    This will enable you to start Cocoon 2.2 from within Eclipse thus many fancy things will
possible (details below)
3. Checkout Cocoon's trunk from
4. Watch following video to see what else should be done:
5. *Before* you start mvn eclipse:eclipse command do following:
    (assuming you are in the root directory of Cocoon checkout)
    $ cd blocks/cocoon-ajax/cocoon-ajax-impl/src/main/resources/org/apache/cocoon/dojo/resources/
    $ unzip
    $ mv dojo-0.4.3-ajax/* .
    $ rm -r dojo-0.4.3-ajax

    Then return to the root directory and continue with the video. If you wonder why this
ugly thing 
is needed: We are cheating here a little bit. It's a build that takes care of unpacking dojo
placing it at the right location but we don't want to rebuild right?
7. Once you are finished with the video you should be able to hack 2.2 *without* any rebuilds.

1. Thanks to the fact that Eclipse knows about Cocoon projects you can easily debug Cocoon.
Just use 
debugging button instead of Run. Nice thing is that it even supports HotSwap so if you change
a java 
class while debugging it will get redeployed. This is what we call RAD, right?
2. I had to modify (touch) forms-samples-styling.xsl file because Cocoon/XSLT processor does
detect changes to the included files. Anyway, AFAIR the same problem is in 2.1.

I hope that this is impressive enough way of convincing you that you are wrong. ;-)

> ... but I am still bogged down with subtle differences in format 
> interpretation between Java and Dojo, with validating number fields, 
> it's a minefield ...... blog entry half written ;)

Eager to read.

>> I see only two small obstacles:
>> 1. As I have already seen it at ApacheCon you have some nice work in 
>> your computer. The problem is that if you keep it on your computer 
>> then nobody can test it and eventually help you with this stuff. Any 
>> reason to not commit your work that you already have to some public 
>> place?
> There are a few problems that have stopped me doing this so far :
> 1) too lazy (so far) to set up and maintain some kind of branch/sandbox ;)

Weak excuse I must say, branching blocks (at least in 2.2) is a very easy task. Actually,
we have 
maintenance branches for Forms and Template blocks here:

I could setup branches for Dojo playings there if you wish.

> 2) I cannot commit anything to head yet, because lots of stuff is still 
> completely broken and/or still has to be re-written to the new APIs. The 
> work has already taken me several months, and there are several more to 
> go ..... it is unpredictable how much longer this will take, I'd mess up 
> Cocoon's release cycles .....

Yes, that's why branching is the only option.

>> Otherwise any collaboration is rather difficult.
> Agreed.
> What would you propose?
> The work involves having two or three custom blocks, forms and ajax 
> (atm, I have dojotoolkit as a block).
> If you are serious about getting involved, I'd be prepared to make the 
> extra effort to collaborate.

It's difficult to say if I'm serious because my plans are little bit changeable at the moment.
situation looks like this: The company I'm working at is seriously interested in migrating
Forms to 
Dojo 1.x but now we are busy with migration to 2.2 that nobody knows how long will take. Actually,

we have some estimations and provided everything goes well I'll be working on this migration

On the other hand, we are making an open source here, right? I would much more prefer you
small changes that others can review than coming with big contribution that nobody can check
follow. Moreover, I've seen some users playing with Dojo 1.x and Cocoon 2.2 on our users mailing

list so there are chances that you will get some supporters even if my plans change for some

Therefore, I would suggest to publish your work regardless of my own plans.

>> 2. I prefer to work with C2.2 (trunk) because it's simpler than 2.1 
>> and it's much easier to develop/test anything here. Any chances that 
>> you will switch with your work to trunk?
> You find 2.2 simpler, I find 2.1 simpler :)
> If we could find the right way to collaborate, you can work on 
> 2.2-specific issues, and I can work on 2.1.

Jeremy, I hope that my video will convince you that things with 2.2 are simpler. Another fact
that more and more people interested in Cocoon switch to 2.2 so your work on it will get more
Also, take into the account that in 2.2 we can release blocks independently of the rest so
it's much 
easier release cycles with real work that gets done.

> One of the major problems with 2.2 is the loss of the 'system pipelines' 
> that in 2.1 provide a set of static URIs for loading cforms and dojo 
> resources; coupled to the fact that /someone/ misunderstanding dojo APIs 
> thought it necessary to introduce a resource-path for use by cforms 
> widgets client-side.

It's been me who removed 'system pipelines' in trunk as we have a superior mechanism for serving

block resources: it's servlet: protocol and Servlet Service Framework in general.

When it comes to the resource-path it wasn't me who introduced this variable.

> I can hopefully help you over-come these problems.
> This is the current JS Loader for 2.1.12-dev :
> <script src="/_cocoon/resources/dojotoolkit/dojo/dojo.js" 
> type="text/javascript" djConfig="isDebug: true, locale: 'en_GB', 
> parseOnLoad: true"></script>
> <script type="text/javascript">
> dojo.require("dojo.parser");
> dojo.registerModulePath("cocoon.forms", "../../forms/js"); 
> dojo.registerModulePath("cocoon.ajax", "../../ajax/js"); 
> dojo.require("cocoon.forms.common"); 
> dojo.addOnLoad(cocoon.forms.callOnLoadHandlers);
> </script>
> (ignoring paths to css for now ....)
> We have a system pipeline "/_cocoon/resources/ .... " which is used as a 
> prefix to load dojo from the dojotoolkit block.

Yep, I remember how it was done in C2.1. Now let's have a look at the same fragment in 2.2:
<script src="{$dojo-resources}/dojo.js" type="text/javascript"/>           <!-- load
dojo -->

<script type="text/javascript">                                            <!-- load
forms library -->

   <xsl:value-of select="concat('dojo.registerModulePath(', $doubleQuote, 'cocoon.forms',

$doubleQuote, ', ', $doubleQuote, $forms-resources, '/js', $doubleQuote, ');')"/>
    <!-- tell dojo how to find our forms module. NB: (since 2.1.11, replaces cocoon.js)

     dojo.require("cocoon.forms.common");  <!-- tell dojo we require the commons library

Apart from the fact that there is an ugly concation these snippets look similar.

> Then we register two modules, forms and ajax, using a path that is 
> relative to where dojo was loaded from.

The problem is that in 2.2 every block can have an arbitrary location where it's mounted so
paths across different blocks won't work. This is a consequence of the fact that block can
*extended* and extending block must have different mount path.

Cocoon 2.2 is less monolith so you can make less number of assumptions but fortunely enough
there is 
one thing coming to yours rescue: it's servlet protocol.

You can just write:
<script src="servlet:dojo-block:/path/relative/to/dojo.js" type="text/javascript"/>

Where this path is relative to block's root directory and rest will be handled by Cocoon.
The nice 
thing about using servlet: protocol is that it's aware of inheritance. So for example, you
create your-fany-dojo-block and declare dojo-block as a super block for it. Then you can override

just a single resource (like CSS) and servlet: protocol will handle it.

Actually, you don't need to use servlet: protocol everywere. It's enough if you use it to
mount path for the block. So there is no trouble with calculated paths in Javascript, etc.

All these things influence how blocks in 2.2 look like but Servlet Service Framework is such
killer feature that it's simply worth small inconveniences.

> One point that was missed by the /someone/ above, was that once a module 
> is registered, you can get a url to it like this :
> var imgSrc = dojo.moduleUrl("cocoon.forms","images/blah.png");
> i.e. it is not necessary to provide it specifically to the client as it 
> is currently done : cocoon.resourcesUri = "<xsl:value-of 
> select="$resources-uri"/>"

As I said earlier, I can agree with this only if you access resources from one block. Anyway,

usually that's the case.

> But TBH, except for a few exceptions like custom data-source urls 
> (dynamic selectionlists etc.) there should be no need to reference 
> anything like this ..... templates should be embedded in widgets, images 
> used in widgets should be loaded via css (where relative references work 
> internally) etc. etc.


> So, the system path is not available in 2.2. The dojotolkit, forms and 
> ajax blocks could have any URI. So we need a standard way for an 
> application block to tell it's form-rendering pipeline the paths to 
> these blocks. Presumably this should be the responsibility of the 
> application's sitemap.
> It should not be necessary to re-write any URIs (!!).

Unfortunately, it's necessary due to reasons I outlined above. But if these things are handled
100% transparent way then is there any problem?

> Furthermore, this provision of paths to blocks, needs to take into 
> account the fact that in production people will most likely want to do 
> stuff like :
> 1) acquire dojo from CDNs like AOL, Google etc.

This is more interesting. To be honest, I have no idea how this could be *easily* handled
by 2.2.

> 2) build custom minimised JS libs to support their apps
> 3) load their own custom modules, override css etc.

This is a damn easy thing in 2.2. That's why I see development on top of 2.2 /easier/ compared
to 2.1.

> 4) lots of stuff we have not thought of yet ;)
> ATM, while I am developing cforms, my dojotoolkit block is a special 
> build, everything uncompressed, unpackaged, etc. with like 180 sets of 
> locales etc. etc. Some complex forms are loading over 100 separate assets.
> The modularity of dojo (and by using dojo.require) means that only what 
> is needed by a page is loaded, which is great. But in production, you 
> will want to heavily reduce the number of files ..... specially the 404s 
> you get 'hunting' the locale tree. It is a bit of a contradiction .....

I don't really follow you here. Could you explain the fragment about 404s and 'hunting' the
locale tree?

> I have not really begun to think seriously about how this should be done 
> yet.

In 2.2 I can see that all these things are done following way:
We ship reasonably standard dojo/ajax block on which Forms block depend on. When someone wants
customize it by overriding resources or by reducing number of files he creates his custom
dojo block 
that *extends* our standard dojo block. Then one can override any pipeline from standard dojo
serving resources by it's own implementation. One can override pipeline's behavior only for
resources and fall-back to standard block for the rest.
The last step is to tell Forms to use customized dojo block which is easy and involves setting
property so original Forms block don't have to be touched at all.

All these magic is already implemented and works well in 2.2 but sadly enough most of people
seem to 
be not aware of it. The reason is rather simple: we don't provide any good sample showing
how this 
works. I guess this is something that should be fixed quickly.

> If we could collaborate on a way to cleanly solve this, so that ideally 
> the basic technique is the same for both 2.1 and 2.2, that would be 
> really useful for me :)

You probably already got the impression that I try to convince you to forget about 2.1 and
switch to 
2.2. :)
I wholeheartedly believe this is a right thing to do but I'm not sure about your feelings.

Unfortunately, Forms block in 2.1 and Forms block in 2.2 are rather far away from themselves
this distance is going to be bigger but not smaller in the future.

>> I've tested it (combined with fix from COCOON-1917) and on the server 
>> side everything looks correct now.
> Great !!!
>> The only problem is that browser sometimes does not behave correctly.
>> I noticed that sometimes when I enter non-latin characters to the text 
>> field they get escaped by a browser.
>> So when I enter something like:
>> światło
>> the browser posts to the server such value:
>> &#347;wiat&#322;o
> Yes, I see this a lot.
> I also see UTF-8 encoding like this : %E2%82%AC (which is the 3 byte 
> encoding for the Euro symbol).
> I have not found this encoding to be a problem.

I thought that browser should never escape characters if it's not absolutely necessary. If
was submitting the data using UTF-8 then that wouldn't be necessary right?

> What problem does this cause you?

Our samples are simply broken that's the problem :-)
If you try to use this upload sample I've already pointed you to then you will see that result
produced after forms is submitted contains escaped characters.

>> (additionally there is parameter: dojo.transport=xmlhttp)
> This is one of the standard parameters that CForms has to add to form 
> submits.
> CForms uses 3 different transports, depending on context:
> 1) ajax-off : normal whole page submit
> 2) ajax-on  : xmlhttp
> 3) ajax-on + form contains a 'file' field : iframe-transport
> Unfortunately, the response to each of these needs to be serialized 
> differently, hence the need to a very complicated sitemap for cforms and 
> this special parameter.

I see. Still I don't understand why our samples are broken.

>> Since I don't know how these things are handled on the client side I'm 
>> not sure how to fix it.
>> Any ideas?
> I need more details of what problem it causes ....

I hope you can reproduce it with upload sample we have in Cocoon.

Best regards,
Grzegorz Kossakowski

View raw message