This blog updates the roadmap for jetty-6, jetty-7 and jetty-8 with the
latest plans resulting from the move to the Eclipse Foundation and the
delay in the servlet-3.0 specification. Previously it was intended that jetty-7 was going to be servlet-3.0, but with the move to eclipse and with the delay of JSR-315, it was decided to delay servlet 3.0 to Jetty-8 later in this year.  Thus the current active branches of jetty are:

Jetty-6 @ codehaus & mortbay

The current stable branch is jetty-6 for servlet-2.5 and java-1.4 (some modules are 1.5).  It is in the org.mortbay.* package space and is licensed under the apache 2.0 license.   However, it is now mostly in maintenance mode and new features will not be added to jetty-6 unless there is compelling reasons to do so.  It includes support for both HTTP server and client and comes bundled with a cometd server.
Jetty-6 is the release for established, production-ready projects.

Jetty-7 @ eclipse

The current development branch is jetty-7 for servlet-2.5 and java-1.5.  It is in the org.eclipse.jetty.* package space and is licensed under both the apache 2.0 and eclipse 1.0 licenses and may be distributed under the terms of either license.  Jetty-7 represents a moderate refactor of the jetty code base:

The intent of jetty-7 is to allow users to transition to the updated architecture and to access some servlet-3.0 features, within a servlet 2.5 container and without the need to update java 1.6 or to wait for the final specification later this year.  There are milestone builds of jetty-7 available already and we hope to have an official eclipse release in the next month or two.
The cometd client and server are now in the cometd.org project and are built against jetty-7. Some jetty integrations (eg jetty-maven-plugin, terracotta,
wadi, etc) and distributions (eg. deb, rpm, hightide) will remain at
codehaus and are now built from codehaus trunk.
Jetty-7 is the release for projects starting development now.

Jetty-8 @ eclipse

The current experimental branch is jetty-8 for servlet-3.0 and java-1.6. It is in the org.eclipse.jetty.* package space and is licensed under both the apache 2.0 and eclipse 1.0 licenses and may be distributed under the terms of either license. Jetty-8 is being kept in lock-step with jetty-7 as much as possible, so that it represents essentially the same server, but rebuilt with java-1.6 and using the standard servlet-3.0 to access the features already available in jetty-7.
Jetty-8 is the branch for people who wish to experiment with the emerging APIs now.

Webtide @ JavaOne

If you want more information about what exactly are these jetty and servlet-3.0 features, why not come to JavaOne 2009?! Webtide will be have a small booth in the expo (where you will mostly find me) and Sun have invited me to participate in their technical
session on Servlet 3.0 at JavaOne, together with Rajiv Mordani and Jan
Leuhe.  I’ll be presenting a section on the Asynchronous Servlets API
and giving a demonstration that uses some ease-of-deployment features
to deploy a webapp on glassfish using the Jetty asynchronous HTTP
client  in a 3.0 asynchronous servlet. The session is TS-3790 “Java

I really like the idea behind the HTML5 Websocket API – namely that a datagram model should be used for web application communication rather than a request/response paradigm (this is also the idea behind cometd).  But unfortunately, the proposed protocol to carry websocket traffic is neither a good protocol nor is it well specified.
After failing in an attempt to get the WebSocket protocol improved, I decided to try to define a better solution.  I had intended to work privately on this for a while, but the twittersphere has pointed out an early draft, so I’ve put the work-in-progress on http://bwtp.wikidot.com and I invite review, feedback and collaborators.
So what’s so bad about the Websocket protocol proposal? The main things I dislike are that the document is impenetrable, the protocol inflexible and it is entirely different from other IETF application protocols for no good reason. But rather than throw (more)mud, I’d rather sing the praises of the approach that I have taken:

If you are interested, I encourage you to join the IETF Hybi mailing list and to join the discussion ragarding the bidirectional web.
 
 
 
 

The announcement of Google Wave is a bold declaration of where Google sees the future of the web. Google, unsurprisingly enough, sees the future of the web as a server side paradigm, with dynamic updates being used to drive the thin client model to capture even more of tasks that where once done client side.  Google are extending the server side model of webmail to apply to applications that have been fundamentally client side, such as document authoring, IM and chat.

Some have said that Wave’s use of XMPP represents the death of HTTP, but I think they’ve got the wrong end of the banana! Wave is using XMPP to federate servers together, not clients.  When it comes to client/server communications, Wave is using GWT over good old HTTP, with some push extensions so that a client can get a dynamic view onto a Wave document, which is a fundamentally server side entity.  If anything, Wave has declared that HTTP is king and a near immortal one at that.

Google’s use of XMPP is roughly equivalent to the existing use of SMTP between mail servers. Instead of passing mail documents between servers using a store and forward model, Wave has the servers dynamically collaborating to maintain a live Wave document that contains content, style, history, permissions and private content. The protocols that Wave might put on the endangered species list are SMTP, POP and IMAP (but have any protocols gone extinct? Has a gopher been sighted in the wild recently or only in captivity?).

If Wave is successful (and it certainly looks pretty compelling), then more traditionally client side state is going to be captured on the server side.  This is a great model for google, as it lets them use their massive serverside databases to power their serverside robots like spelly and rosey, which access the vast databases of Google to do contextual spell checking and translation. You will never get such robots running client side and it is services like these that makes Google confident that they can offer better wave servers than anybody else – hence they do not fear opening up their Wave servers to competition.   So Googles’ webmail competitors had better start thinking of compelling reasons that people will want to host their waves on non-Google servers.

Of course for Jetty, Google Wave is just a brilliant story.  To implement a Wave server, you will need a flexible, performant web server that can well support dynamic push content and will affordably scale into your wave clouds (should they be called oceans rather than clouds now?)  Jetty is the ideal Wave server!  In fact because Wave uses GWT, Google AppEngine and links to shindig, it is already based on and/or using the Google services that are based on or use Jetty. 

For our other key project, cometd.org, the picture is a little less clear.  Google Wave does it’s own comet implementation based on long polling using GWT RPC. But Wave reinforces that comet is now a core web paradigm. Any alternative implementations of Wave that do not use the google GWT code base, would do well to look to cometd.org as a core technology.

For SnoracleZero (aka Java One) this year, we are planning a social get together of Jetty users and Webtide clients  8pm Tuesday (June 2).

If you’d like to come along, email javaone@webtide.com and we’ll pick a venue depending on the estimated numbers.  See you there!

In my December 2008 blog, I strongly criticised the Servlet 3.0 JSR-315 process and the resulting Public Review Draft, describing it as a: “poor document and the product of a discordant expert group (EG) working within a flawed process” and of producing a “Frankenstein monster, cobbled together from eviscerated good ideas and misguided best intentions“.
Perhaps because of these harsh words (or more probably in spite of them), JSR-315 has become
significantly less discordant and some good technical progress has been
made.  While I remain somewhat concerned about the  process (Eg we have a Proposed Final Draft while some significant issues have yet to
be resolved and/or prototyped), I’d like to focus on the improved
spirit of the group and highlight some of the technical achievements that have
resulted.

Asychronous Servlets

JSR-315 has made significant progress on asynchronous servlets. The proposal, identified in my update on the Public Review Draft,
to define a specific dispatch type for async requests has been adopted
and that has resulted in a very workable asynchronous servlet proposal.
Once asynchronous dispatches were separated from normal dispatches,
this made irelevent most of the differing opinions about how filters
should apply and if forward semantics should apply. As a result the
methods previously named AsyncContext.forward(…) have now been renamed to AsyncContext.dispatch(…) and there is general agreement and support of the different asynchronous style of  usage possible with this API.
While
I think the final async proposal is far from perfect and perhaps over
complex, I don’t think any of the proposals (including my own) could
perfectly retrofit asynchronous behaviour to the servlet spec. The
benefit of the complexity is that the proposal well supports multiple
asynchronous paradigms and usage styles.  Most of my prior complaints
were more about specific usage styles and are thus not so important if
multiple styles are well supported.
Without a single imposed asynchronous model and
there will be significant
opportunity for frameworks to inovate in providing various asynchronous
models to the developer community. To this end, the Jetty continuation
style of jetty-6 has been updated in jetty-7 with ideas from
servlet-3.0 and should now be seen as a framework that builds upon the
servlet 3.0 capabilities.

Annotations and pluggability

Some of the key new features of servlet 3.0 is the increased support for ease of deployment with new ways to discovery, configure and deploy Filters and Servlets:

• Annotated filters and servlets may be deployed without the need for a web.xml entry.
• Jars may contain /META-INF/web-fragment.xml
  files with a subset of web.xml configuration.
• Programmatic
  configuration of Filters and Servlets from ServletContextListeners, which are potentially discovered in /META-INF/*.tld files within jars.

Since the Public Review Draft, an additional feature has been added for automatic discovery of webapplication configuration:

I believe these mechanisms are good improvements in the specification and I support their inclusion. I previously expressed concerns about the flexibility and optionality of their usage. Specifically that:

The Proposed Final Draft (8.2.2 & 8.2.3) has addressed all but the last of these concerns with the ability to specify in web.xml an absolute ordering of jars within WEB-INF/lib/ that allows jars to be excluded from the ordering. Each jar in WEB-INF/lib may be given a name by having a <name> element within a META-INF/web-fragment.xml file. The webapps WEB-INF/web.xml file can then have an <absolute-ordering> element that lists the fragment names in the order they will be applied, together with the optional <others/> element to specify if and when unnamed jars are included.
As well as my ordering concern, this feature addresses accidental deployment, as a deployer can list only known well trusted jars; and slow deployment, as an ordering can exclude jars that need not be scanned (other than to discover any web-fragment.xml file).
However, my lingering concern is that the PFD as written does not well express that Filters and Servlets cannot be configured by annotations or TLD listeners from jars excluded from the ordering, nor is it clear that ServletContainerInitializers can be excluded in this fashion.   For the purposes of avoiding accidental and/or slow deployment, it does not matter which of the mechanisms a jar uses for ease-of-deployment, exclusion should mean exclusion.  The responses from the servlet expert group have generally been in agreement with this, but I think the specification needs to be clearer. I have proposed that the following text be added to section 8.2.3:

Conclusion

Despite being in Proposed Final Draft, I think we are not quite at the conclusion stage. However excellent progress has been made and work is continuing. I hope that P in the current PFD is significant and that there will be at least one more draft before we are final. There is still a little time to send your own thoughts to to jsr-315-comments@jcp.org.
Sun Microsystems has invited me to participate in their technical session on Servlet 3.0 at Javaone, together with Rajiv Mordani and Jan Leuhe.  I’ll be presenting a section on the Asynchronous Servlets API and giving a demonstration that uses some ease-of-deployment features to deploy a webapp on glassfish using the Jetty asynchronous HTTP client  in a 3.0 asynchronous servlet. The session is TS-3790 “Java

For the last couple of jetty @ eclipse releases I have been working with getting a bit more used to the eclipse way of doing things. One of those ‘eclipse way’ deals is the p2 update site and I am announcing here the availability of a certain subset of jetty features that are available for download from within eclipse. What does this actually mean?
Well, I am also working on an eclipse plugin or two that ultimately would make use of these features so I went through the process of figuring out this deployment mechanism. I also worked on a updating the osgi HttpService for use with the jetty7 artifacts as well so that was another use I had for it. But ultimately, part of joining eclipse was to bring the jetty components to a larger audience and these update sites seem to be one of the major deployment mechanisms for software within eclipse. Pulling from this update site will not given the average user a new deployment mechanism for their webapp development, that is one of the plugins I have knocking around on the machine which I’ll try to get out pretty soon. What this will give you is either the jetty server, the asynchronous jetty-client, or the jetty servlet components built on top of the jetty server feature. These are functionally repackaged artifacts from the maven repository where we have traditionally deployed our artifacts, renamed into the more eclipse conventional naming and masked into ‘features’. I am very interested in getting feedback on this and I would like to engage directly with developers that make use of these artifacts to both make sure that the update site itself is of the right format (and working against the correct target platform, ouch what a pain that was) and then also to make sure that the feature breakdown makes sense. If this preliminary setup works then I’ll run with it and get the rest of the jetty artifacts we ship (JMX, JNDI, etc etc) into separate features and get them deployed up to the update site.
Anyway, the update site for the last 7.0.0.M2 release is located at:

http://download.eclipse.org/jetty/stable-7/update/

Anyway, feel free to email me feedback directly to ‘jesse DOT mcconnell AT gmail.com’ or you can leave feedback as a comment on this blog, your choice. Your feedback is welcome and will help us get this right for the eclipse developer community.

JJ Zhuang of Yahoo!’s Zimbra division has posted up the architecture of their offline user client. It makes innovative use, or should I say re-use of their server code to present a seamless experience for desktop users regardless of connection state.

He says they did this, with Jetty underpinning, for 3 reasons… reuse, reuse, and reuse.

Read his post here and let us know if we can help your innovative uses of Jetty as well.

I’ll be giving a webinar on Jetty @ eclipse  April 27 – 1:00 pm PDT / 4:00 pm EDT / 8:00 pm GMT.

The presentation will be a mixed bag of some project overview/status, some drill downs into some key features and some hopefully cool demonstrations.

For details visit http://live.eclipse.org/.

cheers

Hot on the heels of Google Widget Toolkit(GWT) switching to Jetty, the little server that can has received some more Google luv’n!   Google’s new App Engine Java service is powered by Jetty! With App Engine, you can build web applications using standard Java
technologies and run them on Google’s scalable infrastructure.
Initially it is a little difficult to see Jetty in use, but you can see the jetty classes in the SDK download and if your application throws an exception then investigation of the stack trace in the log reveals the Jetty servlet container is used.
Not only that, the stack trace and other documentation show that Google have really exploited the embeddablility and extensibility of Jetty at a number of levels:

• They are using their own RPC style connector to receive requests from their front end web servers.
• They use the google account authentication as the only supported authentication mechanism.
• The HTTP sessions are clustered via the database or memcache
• There is an appengine-web.xml configuration file
• Jetty is embedded in their SDK and eclipse development plugin.

Thus Google have plugged in many new and/or extended components in a way that validates our open, component based architecture.   When it comes to application servers we do not believe that one size fits all and strongly encourage such customization for purpose.
We are really pleased that the google App team picked Jetty for their hosting service and welcome them to the ever growing list of Jetty powered projects!
 
The revealing exception:

javax.servlet.ServletException: This is an exception	at com.acme.HelloWorld.doGet(HelloWorld.java:48)	at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)	at org.mortbay.jetty.servlet.ServletHolder$SingleThreadedWrapper.service(ServletHolder.java:617)	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)	at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)	at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)	at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:237)	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)	at org.mortbay.jetty.Server.handle(Server.java:313)	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)	at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:63)	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)	at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:125)	at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:235)	at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4547)	at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4545)	at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)	at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:359)	at com.google.net.rpc.impl.Server$2.run(Server.java:792)	at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56)	at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:489)	at com.google.net.rpc.impl.Server.startRpc(Server.java:748)	at com.google.net.rpc.impl.Server.processRequest(Server.java:340)	at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:422)	at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319)	at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290)	at com.google.net.async.Connection.handleReadEvent(Connection.java:419)	at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:733)	at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207)	at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:101)	at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:249)	at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:373)	at java.lang.Thread.run(Unknown Source)

Edited 11 Apr 2009:
In response to the questions about continuations, we found this discussion on the google app engine forums, which says:

“We do not support continuations.  All requests need to complete their work within the ~30 second deadline and then return control back to App Engine. There is not currently any support for chunked transfer encodings, hanging gets, or background processing, so continuations would not be very useful.”

This is interesting, but we’d like to feedback to google that there are plenty of interesting uses for continuations that do not need waits longer than 30 seconds, background processing or chunked responses:

• We encourage the use of long polling techniques that always send complete responses and thus don’t need chunking etc. 
• In many cases, background processing is not needed. Suspended requests can be resumed by other requests (eg chat, auctions, collaborative editing etc.) or even other responses completing (eg quality of service filter).
• For interactive web applications, having a long poll of 20s will still give great latency and throughput advantages over a more frequent polling solution.

Now it maybe that Googles RPC connector cannot operate asynchronously, so they are forced to have a thread allocated.  If that is the case, then they should talk to us and we’d be please to teach them how to better scale their infrastructure (we also specialize in teaching egg sucking 🙂
 
 

I have had a lot of inquiries about what is going on with the latest versions of jetty being talked about and what they mean to our users so I figured I would give clearing it up a shot.
Greg has recently updated the about page on the jetty eclipse site with some very handy historical (and bleeding edge current) information. Throughout the history of java, jetty has been somewhat a fixture as you can see in the table below, paying particular attention to the rather amusing Status column.

What we see is that jetty has done a pretty solid of performing its major version updates with the updating of the servlet specification, also updating the JDK and JSP versions accordingly. Jetty was well on that track with version 7.0 previously being the first pre-releases of jetty with support for the servlet 3.0 specification, but that has been derailed a bit with the move to eclipse coupled with the postponing of the servlet 3.0 spec until at least the September time frame. Jetty7 has been under development for going on a year next month and it is hardly fair to delay the release of jetty7 and all of the work that has gone on with it until the latter part of the year, much less hold off on released eclipse jetty bundles that long. Jetty pre-release artifacts have been available for a long time and we have been unable to start the release process on it at all because we have been chasing the updated servlet spec. Now that the core of jetty is at Eclipse we need to start the jetty release process as soon as possible to get a stable release available for our users that contains all of the latest and greatest improvements to both packaging and code.
We have taken the opportunity to reorganize the packaging of jetty a bit in the migration of jetty to eclipse and the culmination of these changes are available in the first of our jetty milestone releases 7.0.0.M0 which went out last week. Two of the primary changes from previous jetty7 pre-releases are the package changes and the artifact decomposition. The packaging changes consisted of the obvious org.mortbay -> org.eclipse changes but also include the alignment of maven artifact and java package references. Whereas before in jetty6 the org.mortbay.jetty:jetty-util package may have included classes in org.mortbay.component, org.mortbay.thread and org.mortbay.util packages in jetty7 and future versions of jetty will adhere to the convention of all classes in the org.eclipse.jetty:jetty-util artifact being nested under org.eclipse.util package space. This makes using the existing osgi maven tooling ever so much easier to work with and allows us to work in osgi classloaders without a lot of jury-rigging. In fact we were able to pull out the majority of the custom configuration on the org.apache.felix:maven-bundle-plugin and use the default configuration.
We were also able to wire in generic support for version ranges in the Import-Package: and Export-Package: manifest entries such that the bundles are automatically configured for [7.0,8) which should allow us better future support for working in osgi and eclipse (more on this in a future post covering the osgi http service and some jetty eclipse plugins I have been working on).
As to the artifact decomposition I mentioned, this has to do with cleaning up the transitive dependencies for the using jetty. Jetty is not simply a http server with servlet engine, we also have a robust jetty-client implementation that leverages the solid async implementation under the covers of the jetty server (the soup that lets jetty server scale up to 20k connections). The origin of the client was simple, we needed something on the client side that we could scale up to levels needed to test the server, so jetty-client was born. It is now being used in many interesting places, from proxy server setups to backing the artifact downloads within the maven mercury project. Anyway, the interesting annoyance the was in jetty6 regarding the jetty-client was that it has dependencies on the jetty-server and the servlet-api. Which makes sense given its origins, but was still bothersome to take on another few hundred k of dependencies when all you wanted to use was the client. This has been addressed by the creation of the jetty-io and jetty-http artifacts in jetty7 at eclipse which are shared dependencies of both the jetty-client and the jetty-server and the servlet-api is not longer a dependency of the client. Pretty straight forward stuff but still very important.
If you are one of the people that has been tracking the development of the org.mortbay flavor of jetty7, the jetty7 that is located at eclipse should not be fearful as the changes are not overwhelmingly complex at all. Most of the porting work I have done module wise is taken care of in literally minutes and if you have dependencies on the servlet 3.0 api then jetty8 will be addressing those issues in short order. Jetty7 and Jetty8 will be fundamentally the same codebases with jetty8 containing a thin layer of implementation for the servlet 3.0 api changes on top of the already stable jetty7 core codebase. Jetty7 has been stable and ready to being the release process for literally months while we anticipated the move to eclipse and its packaging changes and the release of the servlet 3.0 api. The only thing holding us back on its release has been the eclipse move and the servlet spec release, heck our plans had originally been to release 7.0 on the day of or the day after the 3.0 spec was released anyway. So now we are in the position to get jetty7 released with the servlet 2.5 support that has been supported since its inception last spring and allow our users to start the process of migrating to the latest stable codebase without the rush to support servlet 3.0 spec.
In addition the forthcoming jetty8 milestone or pre-releases will be taking the place of the jetty7 pre-releases that previously allowed intrepid folks the ability to work with the developing servlet 3.0 api.
Lastly, now that jetty7 and jetty8 are announced and the path for their release should be clearer to all we will start shifting the jetty6 codebase towards a slower maintenance release cycle as we focus on getting jetty7 into the stable category on the chart above. Maybe then we can update the about page again and make jetty5 ‘venerable’ or ‘respected’ and jetty6 ‘mature’.