The Java Community Process has proposed JSR-315 to consider the 3.0 servlet specification. In this blog entry, I look at the Async and Comet considerations listed in the JSR-315 and discuss how 3.0 servlets might address them.
From JSR-315 – Async and Comet support:
- Non-blocking input – The ability to receive data from a client without blocking if the data is slow arriving.
- Non-blocking output – The ability to send data to a client without blocking if the client or network is slow.
- Delay request handling – The comet style of Ajax web application can require that a request handling is delayed until either a timeout or an event has occurred. Delaying request handling is also useful if a remote/slow resource must be obtained before servicing the request or if access to a specific resource needs to be throttled to prevent too many simultaneous accesses.
- Delay response close – The comet style of Ajax web application can require that a response is held open to allow additional data to be sent when asynchronous events occur.
- Blocking – Non-blocking notification – The ability to notify push blocking or non-blocking events. Channels concept – The ability to subscribe to a channel and get asyncronous events from that channel. This implies being able to create, subscribe, unsubscribe and also apply some security restriction on who can join and who cannot.
Non-blocking Input
Undoubtedly modern HTTP servers need to be able to receive request content without blocking. Blocking while waiting for request content consumes threads and memory that can be better used servicing requests that have already been received.
But is non-blocking a capability that needs to be exposed to the servlet developer? Are developers going to be able to do anything valuable with 10% of a XML document, 31% of a SOAP request or 1 byte of a multi-byte character? Do we really want servlet developers to have to deal with all the complexities of asynchronous event handling?
Instead of exposing asynchrous IO to the developers, I believe this concern is better addressed by allowing
servlet containers to do the asynchronous IO and only call the servlet once the entire content is available. Jetty already has this option for content of known size that will fit within the header buffer. Jetty can receive small requests asynchronously and will only dispatch the request to a servlet once the entire content is available and thus the servlet will never block while reading input.
To standardize this approach, there would need to be a way for a servlet to indicate that it wanted the container to perform the IO of request content. The container would only dispatch to the filters and servlets when all the content is available. The content could be made available to the servlet via either the standard getInputStream() API or perhaps via a new getContent() API that could return a byte array, a CharSequence, a File, a DOM document or even an object representing multipart form content (ie file upload).
The concern with this approach would be that large content could consume too much memory if it is not streamed. However, if the container was performing the IO, it could decide to store large content as a file. If there really is a use-case for a servlet to handle streamed data, then I would suggest that the container should aggregate bytes into partial char sequences and parse them into a stream of SAX events passed to the servlet. A servlet developer could handle a SAX event far better than 3 bytes of a 6 byte unicode character!
In summary, to deal with the issues raised by blocking IO, I think that capabilities should be added to the container rather than lower level IO events be exposed to the servlet developer.
Non-blocking Output
My comments about non-blocking Input all apply to Output, only more so! Servlets are application components, and application developers think procedurally for the most part when generating content. If a write returns saying 0 bytes are written, then what is an application code going to do? Wait? Retry? Do something else? More importantly, if that content is being generated from a DOM, JSF, XLS or any famework, then the servlet developer will not have the opportunity to do anything if the write returns with zero bytes written.
Again I would advocate trying to make the container do the asynchronous IO rather than the servlet developer. This can be done now in Jetty simply by providing big buffers, that get flushed asynchronously after the Servlet.service(...) method has completed.
To generalize this, perhaps there should be a Response.sendContent(Object) method, with which a servlet could pass a File, a DOM document or similar. Once the servlet has completed, the container would then take responsibility for generating bytes and asynchronously flushing them to the client.
Again in summary, to deal with the issues raised by blocking IO, I think that capabilities should be added to the container rather than lower level IO events be exposed to the servlet developer.
Delay Request Handling
The ability to delay request handling is very important for dealing with use-cases such as:
- Ajax Comet applications that wait for client specific events before generating a response.
- Servlets that must wait for limited resources such as connections from a datasource.
- Servlets that must wait for asynchronous events such as webservice call responses or proxied request responses.
The key word in these use-cases is “wait” and thus I think “delayed request handling” is not the correct description of this concern. You cannot delay handling of a request without first partially handling the request:
- The request can be authenticated and authorized
- The state can be examined to determine if further request handling must wait for an event
- Requests to slow services may be issued/sent
- Allocation of scarse resources may be registered
So in reality, request handling is not delayed, but is suspended. Handling of the request needs to commence and
progress to a point where it is suspended and then resumed when the criteria for the wait are met. Asynchronous servlet API’s from BEA and Tomcat, address this by special calls to the servlet that allow request handling to be commenced and then completed. I have blogged about the problems with this approach, which in summary is that these new APIs are not existing servlet APIs, so no existing servlets or servlet frameworks can be used to handle these requests or generate content!
The Jetty server has addressed this concern with Continuations that allow request handling to be suspended (or delayed) from within a call to the normal Servlet.service(...); method. Thus existing frameworks and servlets may easily be used with applications that suspend (or delay) request handling.
I strongly believe that the semantics that Servlet 3.0 needs is suspend/resume of normal request handling. The API for this could be Continuations, but with support of the servlet API, a less controversial approach could be to simply add ServletRequest.suspend(long timeout) and ServletRequest.resume() methods. After suspend is called, the response object could disabled until the service method is exited (similar to after a RequestDispatcher.forward() call). When the timeout expires, or the resume method is called, the service method is simply recalled with the same request and response objects. All the currently specified security, session and JEE JNDI context facilities are available for handling and do not need to be redefined and reimplemented for any new APIs.
An indication of the power of suspend/resume sematics of service calls, is that this approach can be used to emulate Asynchronous servlet API’s but the converse is not true.
Delay Response Close
If suspend/resume semantics are adopted for delayed request handling, then delayed response close is also addressed. A response is closed when a service call exits without a suspend call.
Note that the style of Ajax that relies on delayed close is actually a bit of protocol abuse and is not guaranteed to transit proxies. However it is widely deployed and it should be supported, but not encouraged.
Blocking – Non blocking notification
I’m not clear exactly what this concern means? It could be referring to Channel as in an NIO channel, In which case my comments about asynchronous IO above apply. Alternately it could be referring to Channel in terms of a public/subscribe messaging bus like bayeux/cometd, in which case I am hugely in favour of a standard event mechanism being adopted.
In Summary
The async considerations listed in JSR-315 have indeed captured some significant use-cases that need to be addressed by modern servlet containers. Furthermore, I believe that only relatively minor changes in the API are needed to address the majority of these concerns. My next blog entry will expand on the API changes that I have alluded to in this entry.
