Category: General

  • Jetty, JPL and the Mars Rovers

    In what has to be one of the coolest ever uses of Jetty, JPL are using Jetty for missions operations planning on the Mars rovers!!!

    Khawaja Shams from the JPL Operations Planning Software team dropped by the Webtide booth at EclipseCon and just blew us all away with this news. So now whenever I check on the rovers’ progress, I get a warm and fuzzy glow knowing that Jetty is helping this extraordinary mission succeed.

    It seems to me that the longevity and success of the rover project is due to smart engineering based on well designed components – and that’s something that we continually strive for with Jetty. Our recent move to the Eclipse Foundation has already spurred us on to critically examine every jetty module, excise the cruft, streamline a few things and generally enhance our componentization for jetty-7.

    To paraphrase Buzz Lightyear, for Jetty, its “to Mars … and Beyond!!!!”.

  • i-jetty Release 2.0

    Release 2.0 of Jetty for Android (i-jetty) is now available in src and binary form from
    i-jetty downloads.
    This release is the first to also be available (for free) on the Android Marketplace.
    To download from the Marketplace, you’ll need to have an Android handset. You’ll find i-jetty under the Applications category in the Communication subcategory.
    If you’re using the emulator from the SDK, then you should download the i-jetty-2.0-final-signed.apk
    bundle from i-jetty downloads.
    In this release, we support Android sdk 1.1_r1 and jetty-6.1.16. We’ve moved to using a combination of json, javascript and xhr for rendering the Console webapp rather than generating html. We’ve also improved the contacts management part of the Console webapp, allowing you to upload an image for a contact, as well as edit all their phone numbers, addresses etc. A new feature in this release is the ability to view your images and video and listen to the music stored on your phone via your desktop browser. We’ll be adding more functionality to the new media serving feature next release around.
    Speaking of the next release, I’ll be doing some testing of the newly available SDK 1.5 preview to establish whether or not it fixes some of the outstanding android issues that have been bugging i-jetty, like the Bundle resource problem and the dalvik cache problem which have prevented dynamically downloaded webapps from running on a real handset. I hope to have some info on these two issues in the next couple of days.

  • Google AppEngine uses Jetty!

    Hot on the heels of Google Widget Toolkit(GWT) switching to Jetty, the little server that can has received some more Google luv’n!   Google’s new App Engine Java service is powered by Jetty! With App Engine, you can build web applications using standard Java
    technologies and run them on Google’s scalable infrastructure.
    Initially it is a little difficult to see Jetty in use, but you can see the jetty classes in the SDK download and if your application throws an exception then investigation of the stack trace in the log reveals the Jetty servlet container is used.
    Not only that, the stack trace and other documentation show that Google have really exploited the embeddablility and extensibility of Jetty at a number of levels:

    • They are using their own RPC style connector to receive requests from their front end web servers.
    • They use the google account authentication as the only supported authentication mechanism.
    • The HTTP sessions are clustered via the database or memcache
    • There is an appengine-web.xml configuration file
    • Jetty is embedded in their SDK and eclipse development plugin.

    Thus Google have plugged in many new and/or extended components in a way that validates our open, component based architecture.   When it comes to application servers we do not believe that one size fits all and strongly encourage such customization for purpose.
    We are really pleased that the google App team picked Jetty for their hosting service and welcome them to the ever growing list of Jetty powered projects!
     
    The revealing exception:

    javax.servlet.ServletException: This is an exception	at com.acme.HelloWorld.doGet(HelloWorld.java:48)	at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)	at org.mortbay.jetty.servlet.ServletHolder$SingleThreadedWrapper.service(ServletHolder.java:617)	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)	at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)	at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)	at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:237)	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)	at org.mortbay.jetty.Server.handle(Server.java:313)	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)	at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:63)	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)	at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:125)	at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:235)	at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4547)	at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4545)	at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)	at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:359)	at com.google.net.rpc.impl.Server$2.run(Server.java:792)	at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56)	at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:489)	at com.google.net.rpc.impl.Server.startRpc(Server.java:748)	at com.google.net.rpc.impl.Server.processRequest(Server.java:340)	at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:422)	at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319)	at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290)	at com.google.net.async.Connection.handleReadEvent(Connection.java:419)	at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:733)	at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207)	at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:101)	at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:249)	at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:373)	at java.lang.Thread.run(Unknown Source)
    



    Edited 11 Apr 2009:
    In response to the questions about continuations, we found this discussion on the google app engine forums, which says:

    “We do not support continuations.  All requests need to complete their work within the ~30 second deadline and then return control back to App Engine. There is not currently any support for chunked transfer encodings, hanging gets, or background processing, so continuations would not be very useful.”

    This is interesting, but we’d like to feedback to google that there are plenty of interesting uses for continuations that do not need waits longer than 30 seconds, background processing or chunked responses:

    • We encourage the use of long polling techniques that always send complete responses and thus don’t need chunking etc. 
    • In many cases, background processing is not needed. Suspended requests can be resumed by other requests (eg chat, auctions, collaborative editing etc.) or even other responses completing (eg quality of service filter).
    • For interactive web applications, having a long poll of 20s will still give great latency and throughput advantages over a more frequent polling solution.

    Now it maybe that Googles RPC connector cannot operate asynchronously, so they are forced to have a thread allocated.  If that is the case, then they should talk to us and we’d be please to teach them how to better scale their infrastructure (we also specialize in teaching egg sucking 🙂
     
     


  • Jetty @ Eclipse – Jetty7 and Jetty8 (and Jetty6)

    I have had a lot of inquiries about what is going on with the latest versions of jetty being talked about and what they mean to our users so I figured I would give clearing it up a shot.
    Greg has recently updated the about page on the jetty eclipse site with some very handy historical (and bleeding edge current) information. Throughout the history of java, jetty has been somewhat a fixture as you can see in the table below, paying particular attention to the rather amusing Status column.

    Version Home Java HTTP Servlet JSP Status
    8.x Eclipse,
    Codehaus
    1.6 HTTP/1.1
    RFC2616
    3.0 2.1 Experimental
    7.x Eclipse,
    Codehaus
    1.5,
    J2ME CLDC
    HTTP/1.1
    RFC2616
    2.5 2.1 Stabilizing
    Jetty-6.x Codehaus 1.4-1.5 HTTP/1.1
    RFC2616
    2.5 2.0 Stable
    Jetty-5.x Sourceforge 1.2-1.5 HTTP/1.1
    RFC2616
    2.4 2.0 Mature
    Jetty-4.x Sourceforge 1.2 HTTP/1.1
    RFC2616
    2.3 1.2 Ancient
    Jetty-3.x Sourceforge 1.2 HTTP/1.1
    RFC2068
    2.2 1.1 Fossilized
    Jetty-2.x Mortbay 1.1 HTTP/1.0
    RFC1945
    2.1 1.0 Legendary
    Jetty-1.0 Mortbay 1.0 HTTP/1.0
    RFC1945
    Mythical

    What we see is that jetty has done a pretty solid of performing its major version updates with the updating of the servlet specification, also updating the JDK and JSP versions accordingly. Jetty was well on that track with version 7.0 previously being the first pre-releases of jetty with support for the servlet 3.0 specification, but that has been derailed a bit with the move to eclipse coupled with the postponing of the servlet 3.0 spec until at least the September time frame. Jetty7 has been under development for going on a year next month and it is hardly fair to delay the release of jetty7 and all of the work that has gone on with it until the latter part of the year, much less hold off on released eclipse jetty bundles that long. Jetty pre-release artifacts have been available for a long time and we have been unable to start the release process on it at all because we have been chasing the updated servlet spec. Now that the core of jetty is at Eclipse we need to start the jetty release process as soon as possible to get a stable release available for our users that contains all of the latest and greatest improvements to both packaging and code.
    We have taken the opportunity to reorganize the packaging of jetty a bit in the migration of jetty to eclipse and the culmination of these changes are available in the first of our jetty milestone releases 7.0.0.M0 which went out last week. Two of the primary changes from previous jetty7 pre-releases are the package changes and the artifact decomposition. The packaging changes consisted of the obvious org.mortbay -> org.eclipse changes but also include the alignment of maven artifact and java package references. Whereas before in jetty6 the org.mortbay.jetty:jetty-util package may have included classes in org.mortbay.component, org.mortbay.thread and org.mortbay.util packages in jetty7 and future versions of jetty will adhere to the convention of all classes in the org.eclipse.jetty:jetty-util artifact being nested under org.eclipse.util package space. This makes using the existing osgi maven tooling ever so much easier to work with and allows us to work in osgi classloaders without a lot of jury-rigging. In fact we were able to pull out the majority of the custom configuration on the org.apache.felix:maven-bundle-plugin and use the default configuration.
    We were also able to wire in generic support for version ranges in the Import-Package: and Export-Package: manifest entries such that the bundles are automatically configured for [7.0,8) which should allow us better future support for working in osgi and eclipse (more on this in a future post covering the osgi http service and some jetty eclipse plugins I have been working on).
    As to the artifact decomposition I mentioned, this has to do with cleaning up the transitive dependencies for the using jetty. Jetty is not simply a http server with servlet engine, we also have a robust jetty-client implementation that leverages the solid async implementation under the covers of the jetty server (the soup that lets jetty server scale up to 20k connections). The origin of the client was simple, we needed something on the client side that we could scale up to levels needed to test the server, so jetty-client was born. It is now being used in many interesting places, from proxy server setups to backing the artifact downloads within the maven mercury project. Anyway, the interesting annoyance the was in jetty6 regarding the jetty-client was that it has dependencies on the jetty-server and the servlet-api. Which makes sense given its origins, but was still bothersome to take on another few hundred k of dependencies when all you wanted to use was the client. This has been addressed by the creation of the jetty-io and jetty-http artifacts in jetty7 at eclipse which are shared dependencies of both the jetty-client and the jetty-server and the servlet-api is not longer a dependency of the client. Pretty straight forward stuff but still very important.
    If you are one of the people that has been tracking the development of the org.mortbay flavor of jetty7, the jetty7 that is located at eclipse should not be fearful as the changes are not overwhelmingly complex at all. Most of the porting work I have done module wise is taken care of in literally minutes and if you have dependencies on the servlet 3.0 api then jetty8 will be addressing those issues in short order. Jetty7 and Jetty8 will be fundamentally the same codebases with jetty8 containing a thin layer of implementation for the servlet 3.0 api changes on top of the already stable jetty7 core codebase. Jetty7 has been stable and ready to being the release process for literally months while we anticipated the move to eclipse and its packaging changes and the release of the servlet 3.0 api. The only thing holding us back on its release has been the eclipse move and the servlet spec release, heck our plans had originally been to release 7.0 on the day of or the day after the 3.0 spec was released anyway. So now we are in the position to get jetty7 released with the servlet 2.5 support that has been supported since its inception last spring and allow our users to start the process of migrating to the latest stable codebase without the rush to support servlet 3.0 spec.
    In addition the forthcoming jetty8 milestone or pre-releases will be taking the place of the jetty7 pre-releases that previously allowed intrepid folks the ability to work with the developing servlet 3.0 api.
    Lastly, now that jetty7 and jetty8 are announced and the path for their release should be clearer to all we will start shifting the jetty6 codebase towards a slower maintenance release cycle as we focus on getting jetty7 into the stable category on the chart above. Maybe then we can update the about page again and make jetty5 ‘venerable’ or ‘respected’ and jetty6 ‘mature’.

  • Maven, m2e and the Nexus Repository manager

    Jetty-7 is moving to eclipse, cometd-java is moving from jetty to
    cometd.org and the maven build system for dojo has been moved out of
    the main dojo tree. So I’m involved in some major refactoring of
    three interdependent projects at the same time. This refactoring
    would be impossible to contemplate without the tools support for
    these projects. Maven, m2e and nexus are making my life a lot easier.

    Maven

    Apache Maven
    is a build tool of the

  • Jetty @ EclipseRT

    The jetty @ eclipse project is now in incubation at the eclipse foundation as part of the EclipseRT runtime project. On Tuesday 24th March, the Jetty-7 code base was converted to org.eclipse.jetty package space and checked  into the eclipse jetty svn repository!!!

    What is changing?

    It is only the core components of Jetty that are moving to eclipse. The project will also remain at codehaus and it will be at codehaus the the jetty@eclipse components are packaged into distributions of an application server, potentionally mixing in other OS projects (eg. cometd.org, Atomikos JTA and activeMQ JMS).   Thus Jetty will be able to be consumed as components from either maven or eclipse repositories, or as a complete product from codehaus downloads.

    Some components previously built and released with Jetty are being spun out to their own projects or at least modules with their own release cycles.  The cometd implementation has already moved from the contrib repository to the cometd.org project of the Dojo Foundation.

    Jetty will remain apache 2.0 licensed, but the ECL will be added as a dual license so that Jetty may be used and distributed under the terms of either.

    Why the change?

    The move to eclipse was motivated for a variety of reasons that affect all parties concerned:

    Good for Jetty:

    New Ideas and Requirements

    Change is good! It prevents a project from becoming static and complacent. Exposing jetty to new communities, new requirements and new environments will give us an opportunity to continue to improve jetty or validate that it is still relevant and well targeted.

    Improved process

    The eclipse foundation offers jetty improved processes for building and distributing components. The improvements are both legal and technical. Legally, The IP status of Jetty will be greatly improved by the rigour of the eclipse contribution processes. Technically, the eclipse packaging processes have highlighted a number of issues:

    • Too much software was included in each jetty release. Every release was releasing everything from utility libraries and protocol engines, through the servlet container and to specific servlets frameworks like cometd. Many of these components need their own release lifecycles.
    • The module inter dependencies had some issues like the jetty HTTP client depended on the Jetty HTTP server.
    • Some java package where split over multiple jars, preventing reasonable OSGi usage.

    Community

    The move to eclipse will grow the jetty community. The OSGi and eclipse runtime community, already significant jetty users, will be brought into the process of developing and enhancing Jetty. Hopefully new contributions, contributors an corporate sponsorship will result.

    Publicity

    The move generates publicity for the project, which is good in attracting attention to the latest features and developments.  Publicity also helps the commercial interests that sponsor the development of Jetty and of those who use Jetty.  We would like to use this publicity to increase awareness of the asynchronous HTTP client components with in jetty, so that the jetty project is seen as the source of a HTTP server, HTTP client and Servlet container.

    Good for eclipse:

    New Ideas and Requirements

    The Jetty project and community will bring new ideas and requirements to the eclipse runtime community and through it to the OSGi community. For example, the OSGi HTTP Service has been stuck at servlet 2.2 level for a number of years and the Jetty project is keen to see how we can help move equinox and/or the standard onto a more recent HTTP feature set.

    Validate process

    The extensive processes of eclipse are tested by the contribution of a large project like Jetty. The issues already found and fixed in Jetty validate these processes, while the pain points of the process (ipzilla) reveal where they may need to be improved.

    Community

    Jetty will bring hundreds of thousand of new users to the eclipse community, many of whom would have been unaware that eclipse is more than just an IDE and that there is a foundation behind it and that the foundation has an already significant runtime project.

    Publicity

    Eclipse is much more than the eclipse IDE. The eclipse foundation is an open source community focused on building an open development platform comprised of
    extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle. By joining the eclipse runtime project, Jetty will help improve awareness of the eclipse offerings above and beyond the IDE.

    What next?

    The Jetty 6.x version of Jetty will remain at codehausand will continueu to be maintained and supported for some time.  The Jetty 7 code base has now been moved to eclipse and will initially be focused on providing repackage version of the capabilities of Jetty 6.x. Due to the delays in the release of a stable 3.0 servlet API, Jetty 7 will now implement the 2.5 servlet specification and is targetting a Q2 2009 stable release.  In parallel, a Jetty 8.x development branch will be created that will allow continued development of the servlet 3.0 api version, which will now target Q1 2010 stable release.

    In parallel with the jetty development efforts, the Jetty project team intends to engage the OSGi and eclipse equinox community to work to improve, modernize and standardize the OSGi HTTP service for deploying OSGi bundles as web applications that can access the full features of Jetty without the need to assemble a java EE war file.

    Thanks to all contributors and users of Jetty who continue to make this an evolving and dynamic project.

  • Come Meet 5 of the Webtide Crew at EclipseCon

    March 24-26, Webtide has a booth at EclipseCon in Santa Clara, and is running Birds of a Feather sessions. Have Jetty/servlet, Cometd questions, or just want to actually meet in person? Come on down! 19/03/2009

  • Cometd Acknowledged Message Extension

    With the release of the latest cometd-jetty in Jetty 6.1.15, the oft requested feature of reliable message delivery is now supported by the Acknowledged Message Extension in the jetty server and in the dojo and jquery clients.

    The initial concept of cometd was to provide "web quality" communications between the server and the browser, ie it was not intended to provide a reliable, transactional and/or atomic delivery of messages. The premise was that if your network failed, then messages can be lost. While this is sufficient for many applications, it is not sufficient for many others. Luckily, the Bayeux protocol implemented by cometd provides for an extension mechanism, which cometd-jetty, cometd-dojox and cometd-jquery have now used to provide an acknowledged message mechanism for reliable message delivery.

    Server side

    To enable cometd-jetty support for acknowledged messages, the extension must be added to the bayeux instance during initialization:

      bayeux.addExtension(new AcknowledgedMessagesExtension());

     

    The AcknowledgedMessageExtension is a per server extension that monitors handshakes from new clients, looking to see if they also support the acknowledged message extension and then adds the AcknowledgedMessagesClientExtension to each client on handshake.

    Once added to a client, the AcknowledgedMessagesClientExtension prevents messages being delivered on any request other than a /meta/connect so to prevent the possibility of out of order delivery. The extension also maintains a list of unacked messages and intercepts the /meta/connect traffic to insert and check ack IDs.

    Dojox Client

    The clients side for dojo is provided by dojox/cometd/ack.js which was released in dojo 1.3.0b2 (but can also be applied to dojo 1.2.x). To enable client side the dojo requires mechanism is used:

      dojo.require("dojox.cometd.ack");

    This is sufficient to enable the extension, however it may then be programmatically disable/enabled before initialization by setting the ackEnabled boolean field:

      dojox.cometd.ackEnabled = (dojo.query("#ackInit").attr("checked") == "true");
    dojox.cometd.init(cometdUrl);

     

    JQuery Client

    The client side for jquery is enabled by including the jquery.cometd-ack.js file (bundled with jetty 6.1.15):

      <script type="text/javascript" src="../../jquery/jquery.cometd.js"></script>

     

    Details

    In order to enable message acknowledgement, both client and server must indicate that they support message acknowledgement. This is negotiated during handshake. On handshake, the client sends "ext":{"ack": "true"} to indicate that it supports message acknowledgement. If the server also supports message acknowledgment, it likewise replies with "ext":{"ack": "true"}.

    The extension does not insert ack IDs to every message, as this would impose a significant burden on the server for messages sent to multiple clients (which would need to be reserialized to json for each client). Instead the ack id is inserted in the ext field of the /meta/connect messages that are associated with message delivery. Each /meta/connect request contains the ack ID of the last received ack response: "ext":{"ack": 42}. Similarly, each ack response contains an ext ack ID that uniquely identifies the batch of responses sent.

    If a /meta/connect message is received with an ackId lower that any unacknowledged messages held by the extension, then these messages are requeued prior to any more recently queued messages and the /meta/connect response sent with a new ack ID.

    Demo

    There is an example of acknowledged messages in the dojox chat demo that comes bundled with cometd-jetty.

    To run the demo, download the Jetty implementation of cometd, then:

      cd contrib/cometd/demo
    mvn jetty:run

    Point your browser to http://localhost:8080/examples/chat/ and make sure to check "Enable reliable messaging?".

    Use two different browsers instances to initial a chat session, then briefly disconnect one browser from the network (work offline option will do this). While one browser is disconnected, type some chat in the other browser and this will be received when the disconnected browser is reconnected to the network.

    Note that if the disconnected browser is disconnected for in excess of maxInterval (default 10s), then the client will be timed out and the unacknowledged queue discarded.

  • w00t! netcraft survey results

    The results are in for the Febuary 2009 Netcraft Survey and the news is great for Jetty! In January the sites reported using Jetty was at 258k and this month it is at over 289k! That is an increase of over 31k sites in just a month. Sure results vary from month to month as different hosting solutions change back and forth between competing technologies, but it is always nice to see a solid increase for Jetty.
    For comparison, Netcraft now lists Jetty at about 70 percent of Tomcat, and 120% of Resin. Sometimes we get asked if jetty is used in production deployment…um. YES!
    Anyway, wonderful news for Jetty especially on the cusp of the migration of Jetty7 to the Eclipse Foundation which I’ll give an update on the status for in a subsequent blog entry soon.

  • Jetty-based Video on demand in a 'Kangaroo Cull'

    We have always known Jetty is used far and wide. This is the first instance I know of where a governmental watchdog has shut down a project using Jetty on competitive grounds. Not Jetty’s fault, of course, just an industry concentration consideration. As kindly noted, Jetty was the basis of Project Kangaroo for video on demand in the UK, and Webtide have been proud to provide back end support. Unfortunately, the UK’s Competition Commission decided that “this joint venture
    would be too much of a threat to competition in this developing market and has to be stopped.”
    We can hope for the viewing public, that everyone gets to have great video on demand served by Jetty in the future, as hopefully the broadcasters who had joined forces will still continue individually. 🙂