Portal Syndication with Web Services and Cocoon

-

Web Site Syndication has gained popularity as more and more web sites cross reference each other, not only by a single hyperlink, but also by embedding parts of their content. The idea was pioneered by Netscape with their Rich Site Summary (RSS) -( - - http://www.oasis-open.org/cover/rss.html) +(http://www.oasis-open.org/cover/rss.html) XML format. RSS was developed in early 1999 to populate Netscape's My Netscape portal with external newsfeeds ("channels"). Since then RSS has taken on a life of its own and now thousands of Web sites use RSS as a "what's new" mechanism to -drive traffic their way. - -

- +drive traffic their way.

The current RSS 1.0 standard is an application of Resource Description Framework (RDF) -( - - http://www.w3.org/TR/rdf-schema/). +(http://www.w3.org/TR/rdf-schema/). RDF is a framework for describing and interchanging metadata. The RDF framework is extensible and allows adding new types of entities. It also gives meaning to -resources to enable automated processing of Web resources. - -

- +resources to enable automated processing of Web resources.

RSS is unarguably an example of an organically grown and widely accepted standard. For long it was not endorsed by any of the popular standards committees. Even so it quickly became popular and found a large number of creative uses. Lately though it has reached its limits. There is a demand for more advanced portal syndication which RSS cannot satisfy.

-

Latest generation web portals demand more than simply posting cross linked news stories from RSS. Embedding and personalizing rich content and behavior from remote portals is becoming necessity. Limited success has been achieved through complex and sophisticated backend integration via proprietary or Web Services compliant protocols. Recognizing the growing demand, influential organizations have attempted to develop new languages such as:

- - - - +

Web Services Experience Language (WSXL)

-

-( - - http://www.webservices.org/index.php/article/articleview/345/) - - -

- +(http://www.webservices.org/index.php/article/articleview/345/) +

- "WSXL is a Web services centric component model for interactive Web applications. WSXL is designed to achieve two main goals: enable businesses to distribute Web applications through multiple revenue channels and enable new services or applications to be created by leveraging existing applications across the Web." -

- - - - +

Web Services Inspection Language (WSIL)

-

-( - - http://www.webservices.org/index.php/article/articleview/85/) - - -

- +(http://www.webservices.org/index.php/article/articleview/85/) +

- "The specification allows a Web services provider to publish a WS-Inspection (WSIL) document which lists the services on offer and their corresponding WSDL (Web services description language) files. The convention is @@ -2477,83 +1271,46 @@ common entry point to the web site. This paves the way for future Web services "crawlers" to locate and parse WSIL documents for Web service search engines." -

- - - - +

Web Services for Remote Portals (WSRP)

-

-( - - http://www.oasis-open.org/committees/wsrp/) - - -

- +(http://www.oasis-open.org/committees/wsrp/) +

- "Defining an XML and Web services standard that will allow the plug-n-play of visual, user-facing Web services with portals or other intermediary Web applications" -

- - - - +

Web Services for Interactive Applications

-

-( - - http://www.oasis-open.org/committees/wsia/) - - -

- +(http://www.oasis-open.org/committees/wsia/) +

- "Create an XML and web services centric framework for interactive web applications. The designs must achieve two main goals: enable businesses to distribute web applications through multiple revenue channels, and enable new services or applications to be created by leveraging existing applications across the Web." -

-

While these efforts are certainly worthwhile and promising, it will most likely take years before they pass the filters of real life use before they can claim widespread adoption. All of them ask for a thick infrastructure layer to support implementations. While possible, it is unlikely that mainstream deployment will be achieved instantly.

-

Not all is lost though. Fortunately, there is way to satisfy a large portion of the syndication requirements by applying already established technologies and tools. We will illustrate the architecture of a possible solution using an open source framework for XML Publishing - Apache Cocoon.

-

-( - - http://cocoon.apache.org/index.html) - - -

- +(http://cocoon.apache.org/index.html) +

- "Apache Cocoon is an XML publishing framework that raises the usage of XML and XSLT technologies for server applications to a new level. Designed for performance and scalability around pipelined SAX processing, Cocoon offers a @@ -2561,9 +1318,7 @@ and style. To top this all off, Cocoon's centralized configuration system and sophisticated caching help you to create, deploy and maintain rock-solid XML server applications". -

-

First, let's describe a typical use case scenario: User logs in to a familiar portal and happily surfs about. At some point the user clicks on a link which leads to a strange page. It has the portal logo, even shows the same login id @@ -2571,18 +1326,13 @@ frustration the user gets used to switching back and forth between the two faces of the portal ... while looking for another provider which offers both services in a coherent graphical interface.

-

For those who have never had similar experience, we will give a popular example. Yahoo! Autos -( - - http://autos.yahoo.com/finance.html?refsrc=autos/insurance) +(http://autos.yahoo.com/finance.html?refsrc=autos/insurance) offers an easy to use interactive catalog of cars. However when it comes to insuring an automobile, applying for a loan or buying a car, the web site hyperlinks to a co-branded page of another company. For example Lending Tree -( - - https://www.lendingtree.com/newauto/..... +(https://www.lendingtree.com/newauto/..... ) will show Yahoo! Autos logo at the top of the screen, however the rest of the page looks very different than any other Yahoo! page. All the personalization spoils that a Yahoo! user enjoys are lost as soon as the application for a loan @@ -2591,31 +1341,21 @@ appears when switching between the two sites, which reads "You are about to view pages over a secure connection ...". When added up these "negligible" inadequacies, lead to an overall poor experience, which is certainly not the -original intent of the Yahoo! content producers. - -

- +original intent of the Yahoo! content producers.

Now as we have an idea of how things are not supposed to work, we will show that outsourcing interactive components to a third party site, while preserving the look & feel of the original portal is still possible when done right. As we mentioned Cocoon offers a solution. Since Cocoon is a very sophisticated framework, an indepth analysis of its features is beyond the scope of this text to cover.

-

The latest version of Cocoon is 2.1 and it has a new Web Service Proxy component. It is this component which we shall focus on for the reminder of the text. To follow the rest of the article, it will be useful (but not essential) to have a basic knowledge of Cocoon 2.

-

Combined with the XMLForm (which is not part of Cocoon anymore) component of Cocoon 2 and XSLT, the Web Service Proxy component allows vendors to share interactive content with little effort. The Web Service Proxy takes advantage of @@ -2625,14 +1365,10 @@ remote web application and returns the response back to the sitemap for local styling. Receiving a client independent XML format, allows the local site to pull content and style it with XSLT with the desired Look & Feel.

-

- Q. Ok, styling presentation is easy to understand, but how is a form submitted to the original site? -

-

The XMLForm component is the answer. It uses W3C XForms included in the XML content which allows the end user to directly interact with the remote server through the embedding site. The form markup in the XML content of an embedded @@ -2641,65 +1377,42 @@ the relative URL. The Web Service Proxy then takes this information and re-submits it to the original site. It then reads the XML response and makes it available to the sitemap for styling again.

-

- Q. Hmm ... a typical web application maintains a user session while navigating. How is the containing site propagating the end user session to the embedded site? -

-

The answer is simple. The Web Service Proxy simply hooks to the end user session and automatically starts its own session with the remote site. If the remote site requires authentication, then the developer of the local web site has to pass the user credentials as parameters to the WebServiceProxyGenerator.

-

- Q. What transport protocols are supported? -

-

HTTP 1.0, HTTP 1.1, HTTPS.

-

Below we will illustrate the architecture of the solution with some example code and figures.

-

- - -

- +

- Figure 1 - Architecture of the Web Service Proxy Solution. As opposed to a traditional proxy server, the Web Services Proxy captures user input and allows the web site to remain coherent even when the functionality for some of its components is delivered remotely. -

-

- - -

- +

- Figure 2 - Illustration of the data flow for a composite page. Some of the content is locally constructed, the rest is obtained remotely. Finally the same styling is applied and the user facing page appears consistent. -

-

Now we will show a snippet of the sitemap which employs the Web Service Proxy. Notice its brevity! The Web Service Proxy completely handles the content and navigation logic between the two portals. Only stylesheets are additionally required to translate the remotely retrieved documents into a user friendly format.

-

   <?xml version="1.0"?>
   <map:sitemap xmlns:map="http://apache.org/cocoon/sitemap/1.0">
@@ -2727,45 +1440,25 @@
     </map:pipelines>
   </map:sitemap>

-

- - -

- +

- Figure 3 - Above is a sequence diagram outlining the interaction between the key participants in a syndication session. -

-

- - -

- +

- Figure 4 - Sample screenshot from a remotely enabled application as it appears standalone. -

-

- - -

- +

- Figure 5 - Sample screenshot from the same application embedded in another web application. -

-

The content of the original XML page behind these two screenshot follows:

-

   <?xml version="1.0"?>
   <document xmlns:xf="http://apache.org/cocoon/xmlform/1.0">
@@ -2807,22 +1500,15 @@
     </xf:output>
   </document>

-

The listing above contains markup in the XMLForm namespace. It is a presentation independent way to specify input controls. Being XForms compliant it is easy to learn and use. The XSLT stylesheets used to convert the XML above are very simple and will not be listed here. They can found in the Cocoon 2.1 distribution.

-

The Web Service Proxy component is tightly integrated with the Cocoon framework and is particularly convenient to use in combination with XMLForm to enable syndication of website functionality. With the presented sample, we only @@ -2834,85 +1520,51 @@ to a constellation of opportunities, not possible with the classical Model-2 approach where the business logic is directly tied to a graphical output like HTML.

- - - - +

Have more questions?

-

Look at the online demo available in the Cocoon distribution in the samples: - - - http://{host}:{port}/{contextPath}/samples/proxy/. - -

- +http://{host}:{port}/{contextPath}/samples/proxy/.

Then study the source code and if you still have questions, join the cocoon users email list and ask. If you have ideas for improvement then you are more than welcome to discuss it on the cocoon development email list and eventually submit a patch through the Apache bug tracking system.

-

- Errors and Improvements? If you see any errors or potential improvements in this document please help us: - - - + View, Edit or comment on the latest development version (registration required). - - -

- +

-

Portal Syndication with Web Services and Cocoon

What Is Web Syndication?

Going beyond RSS with Web Services

Web Services Experience Language (WSXL)

Web Services Inspection Language (WSIL)

Web Services for Remote Portals (WSRP)

Web Services for Interactive Applications

Apache Cocoon

Web Services Proxy to the rescue

Conclusion

Have more questions?