First off, I think I like wicket, its a pretty neat framework and it has some pretty solid integration with ajax which is interesting. I have to say though that having spent a lot of time on the whole long polling, ajax push, cometd type of communication approach the whole idea of ajax polling back to a server for a bit of content seems kinda quaint.
Some things I like about wicket are that you work in java for all the pages and models, etc and you still let html pages and css and all that kinda stuff work for those people that like working with them for layout, etc. There is no xml configuration for anything, things are woven together in the java which is pretty nice to work with. I did have a chuckle at Martijn Dashorst’s ApacheCon08 presentation where a major feature of wicket was that there was no xml configuration. Seems like not long ago that was a preeminent feature of coolness in application frameworks.
After playing around with a few wicket examples I had to start looking into how wicket and cometd could work together. Enter the awesome wicketstuff project and the recent efforts of Rodolfo Hansen with the wicketstuff-push project. This is a pretty neat initial foray into meshing two great things together. For the cometd integration, the current status of things is that cometd is being used as a notification mechanism more then anything, not really a conduit for information transfer as in more traditional cometd apps. In the chat example provided, you click on the message and it makes a wicket ajax post back to the server, where the response has listeners attached for cometd notification, the message is taken in and cometd pushes out to all of the attached clients that something has happened and then the browser makes a wicket ajax request to get the updated information.
So its a chatty endeavor, but as for melding the convenience of a more traditional web framework like wicket with cometd its not _terrible_ by any means. It really does highlight one of the fundamentally hard problems with something like cometd in my mind. It is not hard to use cometd and get the benefits of the server push to client when your working with a lot of javascript on the client, that environment is very suitable for processing json based information returned in cometd messages from a server. Javascript is simply awesome for that kind of processing, but when most of your brains (web framework) live on the server is it harder to ‘push’ out that sort of information since formatting and data presentation isn’t specifically managed on the browser. For example, the wicketstuff-push example doesn’t use cometd to bring the actual text of any chats from the server, instead it ‘wakes up’ the client and the client calls back with wicket ajax and gets back dom statements to execute to ‘update’ the chat box. In the normal cometd examples all of that information is getting returned in json markup and then evaluated by javascript.
I am not really sure what the answer is there, perhaps as cometd continues to gain more momentum as a protocol then we’ll end up with generic cometd enabled ajax components for web frameworks like wicket. That might be pretty cool…
Author: admin
-
Cometd + Wicket
-
Scalable BlazeDS with async Jetty
Valery Silaev has pointed me to an article on flex.sys-con.com that demonstrates how the asynchronous features of Jetty have been used by Farata Systems to achieve some excellent scalability results with BlazeDS.
BlazeDS is an open source server side that provides comet style streaming events to a flex/flash client side (but not strictly comet as it is not strictly ajax).
Good work!
-
Safari Caching and Embedding Jetty
I ran across a curious issue with safari recently related to caching. I opened it as an issue with Apple because I think its a nasty little issue, and I added it to the jetty wiki recently but I figured it was a big enough gotcha that I would toss a blurp about it here in the hopes of helping someone running across the issue in the future.
There are a host of good descriptions of how various browsers cache their static content and determine whether or not they need to check for updates for it. The best known and most used is probably the Last-Modified: header coupled with something like Expires: to force content to be either reloaded all the time or not at all by putting it far enough into the future. What was recently noticed was that with Safari (at least version 3.1.2 (5525.20.1)) the Last-Modified: header was not enough to achieve even minimal caching. With Firefox simply returning the Last-Modified: header would cache static content for a while but not so with Safari _if_ the Date: header was missing. So…the Date: header appears to be a required header for Safari to use the Last-Modified: header for basic static content caching.
Now what does this have to do with embedding jetty?
Well jetty is really easy to embed, a few lines and you have a full fledged http server in your application, ready to serve content to your hearts desire. If you start up jetty through the normal distributions you are covered because by default the ‘sendDateHeader’ value is set to true (check the bottom of your jetty.xml). However when embedding jetty, by default it doesn’t send the Date: header, it is an option that you need to set through the api.
Server jettyServer = new Server( 8080 );
jettyServer.setSendDateHeader( true );
Now your served pages will pass back the Date: header and safari can go about whatever caching mechanism it is using without constantly checking if static content you have served has changed. So if your embedding jetty and safari is one of your targeted platforms then it is worth double checking that your sending the Date: header back with your responses. Its a pretty simple thing to do that can cut a lot of superfluous calls from safari browsers if its happening for you.
I can’t emphasize enough the usefulness of applications like wireshark or charles in debugging odd browser/server situations like this, but that is another story. -
ApacheCon 2008 – New Orleans
Well, the blog seems to have moved to its new site successfully so its time to start adding some more entries up now and then. First and foremost I want to extend an invitation to users of jetty to meet up with me at the 2008 ApacheCon in New Orleans in a couple of weeks.
http://us.apachecon.com/c/acus2008/
I’ll have some t-shirts available for people interested in supporting jetty!
While jetty is not an apache project (nor specifically the reason I’ll be there) it has enjoyed immense usage within many apache projects like maven, maven plugins, continuum, hadoop and others. So jetty users that find themselves at ApacheCon please feel free to find me, I’ll probably be hanging around with various maven folks.
cheers! -
Google Android chat using Jetty Cometd java client
I have ported the cometd java client from Jetty to run on the google android phone. Cometd implements the bayeux protocol for push messaging over HTTP and is normally used by Ajax clients running in the browser. By porting the java client to android, this allows native applications on the phone to have bidirectional messaging to a server over HTTP.
To demonstrate this, I’ve implemented a simple chat room that interacts with the Ajax Comet chat room demos. This would be a great basis for other android applications that need to update in realtime.
The following core dependencies were used:- jetty-client-6.1.12.rc2 (asynchronous http client)
- bayeux-client-6.1.12.rc2 (bayeux protocol via http)
- android-1.0_r1
I had been wrestling with android for a couple of days trying to get familiar with their apis. The more you dive into it, you would notice a pattern that you might be used to, which is developing webapps using the MVC pattern .
View = xml resources (like html .. static rendering)
Activity = Controller (logic)To get started, you can use the ADT (Android Developer Tools) eclipse plugin. Although it could speed up your productivity, it would can up your cpu and memory as well. It takes a few seconds to recompile(annoying) even with just the slightest change in code.
The
import part of the plugin is the auto-update of the R.class of your
application (after adding/editing/removing your views and other xml
resources).
Implementing the chat client using cometd-jetty client was a breeze because all the work (bayex protocol) was already done by org.mortbay.cometd.client.BayeuxClient.
The one problem I’ve encountered was updating android’s UI from another thread (e.g message received event). If I had read all the android FAQs, I wouldn’t have been stuck on it. Oh well …so the solution was simply creating an android.os.Handler inside your Activity to be able to receive asynchronous events (dispatched from another thread) and to be able to render/update your UI.
public class MyActivity extends Activity
{
public static final int SOME_EVENT = 10;
Handler _handler = new Handler()
{
public void handleMessage(android.os.Message msg)
{
switch(msg.what)
{
case SOME_EVENT:
// update your ui here
break;
}
}
}
Runnable _someEventHandler = new Runnable()
{
public void run()
{
_handler.sendEmptyMessage(SOME_EVENT);
}
}
…..
}
The source and apk is available here.
To chat live (after installation), change your connection settings:
host: cometdchat.morphexchange.com
port: 80
Enjoy! -
Open Source is Free Software
Open source business plans are tough. It’s hard to come up with a clear and comprehendable revenue model when you give so much of your work away for free.
But those of us who work in open source, must not forget that freedom is the at core of our model and that we must grant those freedoms on a non-discrimanatory basis. Recently I’ve seen several postings that make me concerned that some core principals are being forgotten.
Firstly LWN reported on the Linux Plumbers Conference, where the key-note from Greg Kroah-Hartman tried to shame Canonical for not contributing enough kernel patches back from their work on ubuntu:
"there is the matter of redistributors who base their products on
another distributor’s work; these are distributors like Ubuntu or CentOS.
There are no contributions back to the community from that kind of
distributor at all. They are not functioning as a part of the Linux
ecosystem."Then Rod Johnson blog on springsofts maintenance describes the ideal open source business model as:
"If you are an organization deriving tremendous value from Spring by
using it in large production environments, please send SpringSource a
check for 1% of the value you are receiving by using Spring. We will
use this money to pay salaries, grow our investment in open source and
return a profit."Another example of this thinking comes from the blog of Jack Slocum from Ext.js:
"Ext JS 1.0 is released under the LGPL, minus the Assets license as
mentioned above. Shortly thereafter 2 major publicly traded
corporations (names withheld) embedded Ext JS into their development
frameworks. With no mention of Ext JS except in credits files that no
one ever saw. No support for all the work that had been put into the
framework. Neither one of them even contacted us. How can that be
possible? Can they do that?"While I have some sympathy with Rod’s and Greg’s positions, I do believe they are forgetting something fundamental. The users of our Free and Open Source Software (FOSS) are free from any obligation to contribute back patches or cash. FOSS licenses don’t say "you are free to use this unless you are making money" and they don’t say "legally speaking your not obligated to contribute cash or code, but morally you really should".
What Greg Kroah-Hartman needs to remember, is that contributions are not just code or cash. Growing market share, educating users, finding new uses are all great contributions to an open source project.
What Rod is forgetting, is that spring software is now a free commodity, to be used without cost. The expensive valuable resource that Springsoft should focus their marketing on is the clever people that created spring.
What Jack needs to realize is that major publicly traded corporations are unlikely to integrate closed source, proprietary licensed software into their core infrastructure if it it comes from a small operation without major support infrastructure. My mother frequently asks: "why don’t you just charge $1 per month to all those hundreds of thousands of Jetty users?" and I realize that my mother has never tried to obtain a purchase order from a major publicly traded corporation.
At Webtide, we sell developer advice, custom development and production support for jetty and dojo cometd. We don’t expect our clients to buy our services because of some sort of guilt trip from the value they obtain from those projects. We expect our clients to pay for the value add that we give. The software is free under the terms of the apache 2.0 license and we expect no charity or moral obligation in return. But our developers are highly skilled, and if you want their advice, effort and/or experience directly applied to your commercial concerns, then you have to pay for that valuable resource.
I consider it a confirmation of the quality and value of the project when large corporations make it part of their infrastructure or ship product that contains it. I’m glad that many users can often succeed on their own without questions, or producing bug reports and/or fixes. It indicates that the project is doing a good job of well designing, implementing and documenting our software for those users. When Jetty users are profitable, we don’t see that as a lost revenue opportunity, but as a potential client who already should appreciate the value of the skills we have on offer.
You can’t be half free. For those struggling with open source business models, my (free) advice is to embrace the freedom that is fundamental to the success of your project and don’t resent the success of others who use the freedoms that you have granted. If you want to sell software, then don’t open it or get a job at Microsoft, IBM or Oracle.
-
Asynchronous Restful Webapplication
This blog annotates the Jetty 7 example web application (also updated for jetty-9) that uses Jetty asynchronous HTTP client and the proposed suspendable servlets 3.0 API, to call an eBay restful web service. The technique combines the Jetty asynchronous HTTP client with the Jetty servers ability to suspend servlet processing, so that threads are not held while waiting for rest responses. Thus threads can handle many more requests and web applications using this technique should obtain at least ten fold increases in performance.
The screen shot above shows four iframes calling either a synchronous or the asynchronous demonstration servlet, with the following results:
- Synchronous Call, Single Keyword
- A request to lookup ebay auctions with the keyword “kayak” is handled by the synchronous implementation. The call takes 261ms and the servlet thread is blocked for the entire time. A server with a 100 threads in a pool would be able to handle 383 requests per second.
- Asynchronous Call, Single Keyword
- A request to lookup ebay auctions with the keyword “kayak” is handled by the asynchronous implementation. The call takes 254ms, but the servlet request is suspended so the request thread is held for only 5ms. A server with a 100 threads in a pool would be able to handle 20,000 requests per second (if not constrained by other limitations)
- Synchronous Call, Three Keywords
- A request to lookup ebay auctions with keywords “mouse”, “beer” and “gnome” is handled by the synchronous implementation. Three calls are made to ebay in series, each taking approx 306ms, with a total time of 917ms and the servlet thread is blocked for the entire time. A server with a 100 threads in a pool would be able to handle only 109 requests per second!
- Asynchronous Call, Three Keywords
- A request to lookup ebay auctions with keywords “mouse”, “beer” and “gnome” is handled by the asynchronous implementation. The three calls can be made to ebay in parallel, each taking approx 300ms, with a total time of 453ms and the servlet request is suspended, so the request thread is held for only 7ms. A server with a 100 threads in a pool would be able to handle 14,000 requests per second (if not constrained by other limitations).
It can be seen by these results that asynchronous handling of restful requests can dramatically improve both the page load time and the capacity by avoiding thread starvation.
The code for the example asynchronous servlet is available here (updated for jetty-9 )and works as follows:- The servlet is passed the request, which is detected as the first dispatch, so the request is suspended and a list to accumulate results is added as a request attribute:
if (request.isInitial() || request.getAttribute(CLIENT_ATTR)==null){ String[] keywords=request.getParameter(ITEMS_PARAM).split(","); final List<Map<String, String>> results= Collections.synchronizedList(new ArrayList<Map<String, String>>()); final AtomicInteger count=new AtomicInteger(keywords.length); request.suspend(); request.setAttribute(CLIENT_ATTR, results);The request is suspended before starting the searches in order to avoid races if the searches somehow complete before the request is suspended.
In Jetty-9 version, we just check for results==null to identify if we are processing the initial request or not and the standard Servlet 3.0 startAsync() method is used to suspend the request:// If no results, this must be the first dispatch, so send the REST request(s) if (results==null) { final Queue<Map<String, String>> resultsQueue = new ConcurrentLinkedQueue<>(); request.setAttribute(RESULTS_ATTR, results=resultsQueue); final AsyncContext async = request.startAsync(); async.setTimeout(30000); ... - After suspending, the servlet creates and sends an asynchronous HTTP exchange for each keyword:
for (final String item:keywords){ ContentExchange exchange = new ContentExchange() { protected void onResponseComplete() throws IOException { // see step 3 below } }; exchange.setMethod("GET"); exchange.setURL("http://open.api.ebay.com/shopping?MaxEntries=5&appid=" + ... _client.send(exchange);The API for the Jetty Http client exchanges was inspired by the callback style of javascript XHR.
For Jetty-9 the new asynchronous HTTP Client API is used:for (final String item:keywords) { _client.newRequest(restURL(item)).method(HttpMethod.GET).send( new AsyncRestRequest() { @Override void onAuctionFound(Map<String,String> auction) { resultsQueue.add(auction); } @Override void onComplete() { if (outstanding.decrementAndGet()<=0) async.dispatch(); } }); } - All the rest requests are handled in parallel by the eBay servers and when each of them completes, the call back on the exchange object is called. The code (omitted above, shown below)extracts auction information from the JSON response and adds it to the results list. The count of expected responses is then decremented and when it reaches 0, the suspended request is resumed: xxx
protected void onResponseComplete() throws IOException { Map query = (Map) JSON.parse(this.getResponseContent()); Object[] auctions = (Object[]) query.get("Item"); if (auctions != null) { for (Object o : auctions) results.add((Map) o); } if (count.decrementAndGet()<=0) request.resume(); }For Jetty-9 some of this uses the Servlet API for dispatch instead of resume and moves the add message out of this message, but is essentially the same:
@Override void onComplete() { if (outstanding.decrementAndGet()<=0) async.dispatch(); } - After being resumed (dispatched), the request is re-dispatched to the servlet. This time the request is not initial and has results, so the results are retrieved from the request attribute and normal servlet style code is used to generate a response:
List<Map<String, String>> results = (List<Map<String, String>>) request.getAttribute(CLIENT_ATTR); response.setContentType("text/html"); PrintWriter out = response.getWriter(); out.println("<html><head><style type='text/css'>img:hover {height:75px}</style></head><body><small>"); for (Map<String, String> m : results){ out.print("<a href=""+m.get("ViewItemURLForNaturalSearch")+"">"); ...
This example shows how the Jetty asynchronous client can easily be combined with the asynchronous servlets of Jetty-8/9 (or the Continuations of Jetty-7) to produce very scalable web applications.
-
Bad Robot! Google Android is evil
Webtide has been putting some effort into porting Jetty onto Google’s Android mobile phone platform. We were seduced to expend this effort by the promise from Google that android would provide “a new level of openness”. Yet we may be forced to abandon this effort as Google’s bad robot breaks Asimovs 3 laws of Robotics as they have been modified for openness by the the eclipse foundation.
The core issue, is that after an initial promising start, google appear to have backtracked on the openness bit. SDKs updates are only being distributed to select partners who are gagged with NDAs and thus can’t illuminate the community as to the now unclear intentions of Google. This bad robot violates all three laws:A committer may not, through action or inaction, violate IP cleanliness
While not an IP issue, there is a real issue that google may have infringed the GPL by attempting to limit distribution rights with an NDA. They have been officially inactive to try to remedy this and their unofficial weasling out of it does not satisfy the GPL.A committer may not, through action or inaction, disenfranchise contributors
Webtide has been disenfranchised of the freedom to equally compete with those that have access to the SDK. We were induced to contribute our i-jetty efforts to the android community by the promise of openness and the OSI licenses applied to the initial SDK releases. Those promises have proved hollow and the terms of the licenses used have been disrespected at least in spirit, if not in the letter.A committer may not, through action or inaction, surprise the membership
We are waiting for the surprises that will come with the revelation of the secret APIs in the lastest SDKs. We do know that some of the priviledge few with access to the JDK are experimenting with i-jetty. But we have no idea if the secret APIs that google will not reveal might include HTTP and/or servlet capability and our efforts will be for naught. Thus we feel duped into supporting the development efforts of Google and their partners without the transparency needed to allow us to take this business risk openly.
I see little reason that we should continue to put development resources towards this project. While we accept and embrace that that the vast majority of the users of our software will do so freely, we are not a charity and certainly not to the likes of Google. We give away our software and openly collaborate on it’s development, but if one wants our commercial priorities to be aligned with their own commercial activities, then we expect a commercial relationship. Google have used false open pretenses to induce ourselves and many others to align our open source efforts with their closed commercial priorities. -
Dojo Toolkit Maven Repository
Using maven to build your project is a fantastic for managing your dependencies and avoiding having dependencies (and their dependencies) checked into your own svn. The only fly in the ointment, is projects that don’t publish maven artifacts, and the Ajax dojo toolkit has been one of these. Until now that is !
I have added a maven build to dojo and created maven repositories for snapshots and releases at:These repositories contain artifacts for:- dojo toolkit release as a zip, tar.gz and tar.bz2
- dojo toolkit packaged as a java war file ready to serve. The war includes a filter that sets the Cache-Control header to encourage browser caching.
- cometd artifacts for java API and cometd example war
However, if you are a “from first principals” type, then to checkout and build dojo is now as simple as:svn co http://svn.dojotoolkit.org/src/view/anon/all/trunk dojocd dojo/util/mavenmvn
This will build the standard dojo release and package it as a zip, a tar.gz and tar.gz2 and make them available in your local repository. The groupId is org.dojotoolkit and the artifactId is dojo. The following incantation of the maven dependency plugin will download and unpack these artifacts into your maven project:<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>unpack dojo</id> <phase>generate-sources</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>org.dojotoolkit</groupId> <artifactId>dojo</artifactId> <version>${project.version}</version> <type>zip</type> </artifactItem> </artifactItems> <outputDirectory>${project.build.directory}/dojo</outputDirectory> </configuration> </execution> </executions></plugin>Along with the build of the dojo release artifacts, dojo is also packaged as a war file with groupId org.dojotoolkit and artifactId dojo-war. The war produced by this can be directly deployed and used by other web applications on the same server (there is no requirement for js to be served from the same war as your application). Alternately, this war can be used as an overlay by the maven war plugin and merged with your own war project:<project xmlns="..."> <modelVersion>4.0.0</modelVersion> <groupId>vom.acme</groupId> <artifactId>myproject</artifactId> <packaging>war</packaging> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <configuration> <overlays> <overlay></overlay> <overlay> <groupId>org.dojotoolkit</groupId> <artifactId>dojo-war</artifactId> <excludes> <exclude>META-INF/**</exclude> </excludes> </overlay> </overlays> </configuration> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>org.dojotoolkit</groupId> <artifactId>dojo-war</artifactId> <version>1.2-SNAPSHOT</version> <type>war</type> <scope>runtime</scope> </dependency> </dependencies></project>
Currently only snapshot releases have been deployed to the repository (1.2-SNAPSHOT). Once we have some feedback that this is working OK, I will deploy a retrospective dojo 1.1.1 release and organize for that to be mirrored to the central maven repositories. Until that time, you can access the dojo repositories directly with the following incantation in your pom.xml:<repositories> <repository> <id>dojo</id> <name>Dojo Maven2 Repository</name> <url>http://download.dojotoolkit.org/maven2</url> <releases> <enabled>true</enabled> <updatePolicy>daily</updatePolicy> <checksumPolicy>fail</checksumPolicy> </releases> <snapshots> <enabled>false</enabled> </snapshots> <layout>default</layout> </repository> <repository> <id>dojoSnapshots</id> <name>Dojo Maven2 Snapshot Repository</name> <url>http://download.dojotoolkit.org/maven2-snapshot</url> <releases> <enabled>false</enabled> </releases> <snapshots> <enabled>true</enabled> <updatePolicy>always</updatePolicy> <checksumPolicy>warn</checksumPolicy> </snapshots> <layout>default</layout> </repository></repositories>
I am now using these artifacts to better mavenize the cometd project and it’s examples. Feedback on the repositories and the packaging is most welcome. -
jetty-test-servers
Jan and I have run across some annoying problems with test cases and test scoping within jetty recently and it has lead to some discussion on an idea that I have been kicking around for a while. The problem was that we have some useful test cases which are in the form of jetty server configurations in the jetty module and maybe a couple of other module, but since they are test source they are not readily available for consumption in other modules. We ran into this most recently working with eclipse which has no concept of test scoped source vs artifact scoped source so it was merrily working away while the cli maven build was failing because a particular test source was not available in another module.
I worked around that for a little bit by having the jetty module produce a test-component classified artifact which could be pulled in as a test scoped dependency where it was needed. I offered it out as one approach to solve this type of problem but it has the downside of being another bastard artifact being produced by the assembly plugin and being carried along for the ride with the normal jetty artifact. Its also yet another little bit of configuration in that test-component.xml assembly descriptor which would be nice to do without. It would also be possible to use the test-jar mojo on the maven-jar-plugin which would make another artifact again that has the classifier of ‘tests’ which is a bit more standard but is bringing along a whole lot of other classes that we neither want nor need to pass down the classpath to other test modules. I did just dispense with the test-component is favor of using the maven-jar-plugin so we don’t have that one little bit of extra configuration.
We kicked around the idea of just pulling some of these test servers and classes out of the test scope of the jetty module and simply putting them into a jetty-test artifact, but then we suffer the loss of unit test coverage for the jetty module, which is rather core to jetty and not particularly desirable.
Finally we got back around to the idea of making a jetty-server-test module that contained a variety of server configurations suitable for unit testing. It would be a little bit of work but since jetty is just so insanely easy to embed it would really be a pretty trivial job to make a jetty-server-test artifact that contained things like JettyWebappServer, JettySslServer, JettyWebServer all with simple default configurations and coming packaged with keystores, etc for common testing needs. Then it would be a matter of simply instantiating the server you want, maybe setting a path or three and using it in unit tests.
I guess my question now would be, ‘Is it worth it?’ and ‘Would it be generally useful to people or just us inside of jetty?’
hmm.