Greg Wilkins gave the following session at JavaOne 2014 about Servlet 3.1 Async I/O.
It’s a great talk in many ways.
You get to know from an insider of the Servlet Expert Group about the design of the Servlet 3.1 Async I/O APIs.
You get to know from the person the created, developed Jetty and implemented Servlet specifications for 19 years what are the gotchas of these new APIs.
You get to know many great insights on how to write correct asynchronous code, and believe it – it’s not as straightforward as you think !
You get to know that Jetty 9 supports Servlet 3.1 and you should start using it today 🙂
Enjoy !
Tag: servlet
-
JavaOne 2014 Servlet 3.1 Async I/O Session
-
HTTP/2 Push with experimental Servlet API
As promised on my last post on HTTP/2, we have implemented and deployed the HTTP/2 Push functionality on this very website, webtide.com. For the other HTTP/2 implementers out there, if you request
"/"on webtide.com, you will get"/wp-includes/js/jquery/jquery.js"pushed. We have already implemented SPDY Push in the past, but this time we wanted to go a step further and implement HTTP/2 Push in the context of an experimental Servlet API that applications can use to decide what to resources needs to be pushed.The experimental Servlet API (designed by @gregwilkins) is very simple and would consist of only one additional method in
javax.servlet.RequestDispatcher:public interface RequestDispatcher { public void push(ServletRequest request); .... }An application receiving a request for a primary resource, say
index.html, would identify what secondary resources it would like to push along with the primary resource. For each secondary resource, the application would obtain aRequestDispatcher, and then callpush()on it, passing the primary resource request:public class MyServlet extends HttpServlet { public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String uri = request.getRequestURI(); if ("/index.html".equals(uri)) { String resourceToPush = "/js/jquery.js"; RequestDispatcher dispatcher = request.getRequestDispatcher(resourceToPush); dispatcher.push(request); } } }For applications that use web frameworks, in general, is difficult to identify a resource to push. For example, if you use a JSF library, your application is not in control of what secondary resources the JSF library may need to push (for example, css, javascript snippets, images, etc. associated to the JSF components that are being rendered).
Browsers, on the other hand, are in a much better position to identify secondary resources belonging to a primary resource, when they parse the primary resource. It would be great if browsers could request those resources with a special HTTP header that marks the secondary resource request as associated to the primary resource. Not only, it would be great if this could be completely automated, so that applications need not to worry about primary and secondary resources.
This is exactly what we have done in
PushCacheFilter. We have implemented a strategy where theRefererheader is being used to associate secondary resources to primary resources. With this association information, the filter builds a cache where secondary resources are linked to a primary resource, and every time a primary resource is being requested, we push also the associated secondary resources.The
PushCacheFilterlooks for the resource being requested; if it is not known to the filter, it assumes it is a primary resource and assigns a timestamp to it. Then it “opens” a window of – by default – 2000 ms where other requests may arrive; if these other requests have the former request as referrer, then these are secondary resources associated to the primary resource. The next time that the primary resource is requested, the filter knows about it, and pushes its secondary resources via the experimental Servlet API discussed above.We have kept the filter intentionally simple to foster discussion about what strategies could be more useful, and what features would be needed, for example:
- Would browsers use a special header (not the
Refererheader) to mark the a resource as associated to another resource ? - How would be possible to evict entries from the push cache without manual intervention ?
- Is there a relationship between the cacheability of the primary resource and that of the secondary resources that we can leverage ?
- How can a browser tell the server to not push a resource that it is already in the browser’s cache ?
We encourage anyone that is interested to join the Jetty mailing lists and contribute to the discussion.
If you are interested to make your website faster, look at what HTTP/2 Push could do to your website (from our SPDY Push Demo Video), and contact us.
- Would browsers use a special header (not the
-
Websocket Example: Server, Client and LoadTest
The websocket protocol specification is approaching final and the Jetty implementation and API have been tracking the draft and is ready when the spec and browsers are available. Â More over, Jetty release 7.5.0 now includes a capable websocket java client that can be used for non browser applications or load testing. It is fully asynchronous and can create thousands of connections simultaneously.
This blog uses the classic chat example to introduce a websocket server, client and load test.
The project
The websocket example has been created as a maven project with groupid com.example. The entire project can be downloaded from here.  The pom.xml defines a dependency on org.eclipse.jetty:jetty-websocket-7.5.0.RC1 (you should update to 7.5.0 when the final release is available), which provides the websocket API and transitively the jetty implementation. There is also a dependency on org.eclipse.jetty:jetty-servlet which provides the ability to create an embedded servlet container to run the server example.
While the project implements a Servlet, it is not in a typical webapp layout, as I wanted to provide both client and server in the same project.   Instead of a webapp, this project uses embedded jetty in a simple Main class to provide the server and the static content is served from the classpath from src/resources/com/example/docroot.
Typically developers will want to build a war file containing a webapp, but I leave it as an exercise for the reader to put the servlet and static content described here into a webapp format.
The Servlet
The Websocket connection starts with a HTTP handshake. Thus the websocket API in jetty also initiated by the handling of a HTTP request (typically) by a Servlet. The advantage of this approach is that it means that websocket connections are terminated in the same rich application space provided by HTTP servers, thus a websocket enabled web application can be developed in a single environment rather than by collaboration between a HTTP server and a separate websocket server.
We create the ChatServlet with an init() method that instantiates and configures a WebSocketFactory instance:
public class ChatServlet extends HttpServlet { private WebSocketFactory _wsFactory; private final Set _members = new CopyOnWriteArraySet(); @Override public void init() throws ServletException { // Create and configure WS factory _wsFactory=new WebSocketFactory(new WebSocketFactory.Acceptor() { public boolean checkOrigin(HttpServletRequest request, String origin) { // Allow all origins return true; } public WebSocket doWebSocketConnect(HttpServletRequest request, String protocol) { if ("chat".equals(protocol)) return new ChatWebSocket(); return null; } }); _wsFactory.setBufferSize(4096); _wsFactory.setMaxIdleTime(60000); } ...The WebSocketFactory is instantiated by passing it an Acceptor instance, which in this case is an anonymous instance. The Acceptor must implement two methods: checkOrigin, which in this case accepts all; and doWebSocketConnect, which must accept a WebSocket connection by creating and returning an instance of the WebSocket interface to handle incoming messages. In this case, an instance of the nested ChatWebSocket class is created if the protocol is “chat”.  The other WebSocketFactory fields have been initialised with hard coded buffers size and timeout, but typically these would be configurable from servlet init parameters.
The servlet handles get requests by passing them to the WebSocketFactory to be accepted or not:
... protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { if (_wsFactory.acceptWebSocket(request,response)) return; response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE, "Websocket only"); } ...All that is left for the Servlet, is the ChatWebSocket itself.  This is just a POJO that receives callbacks for events. For this example we have implemented the WebSocket.OnTextMessage interface to restrict the call backs to only connection management and full messages:
private class ChatWebSocket implements WebSocket.OnTextMessage { Connection _connection; public void onOpen(Connection connection) { _connection=connection; _members.add(this); } public void onClose(int closeCode, String message) { _members.remove(this); } public void onMessage(String data) { for (ChatWebSocket member : _members) { try { member._connection.sendMessage(data); } catch(IOException e) { e.printStackTrace(); } } } }The handling of the onOpen callback is to add the ChatWebSocket to the set of all members (and remembering the Connection object for subsequent sends). The onClose handling simply removes the member from the set.  The onMessage handling iterates through all the members and sends the received message to them (and prints any resulting exceptions).
The Server
To run the servlet, there is a simple Main method that creates an embedded Jetty server with a ServletHandler for the chat servlet, as ResourceHandler for the static content needed by the browser client and a DefaultHandler to generate errors for all other requests:
public class Main { public static void main(String[] arg) throws Exception { int port=arg.length>1?Integer.parseInt(arg[1]):8080; Server server = new Server(port); ServletHandler servletHandler = new ServletHandler(); servletHandler.addServletWithMapping(ChatServlet.class,"/chat/*"); ResourceHandler resourceHandler = new ResourceHandler(); resourceHandler.setBaseResource(Resource.newClassPathResource("com/example/docroot/")); DefaultHandler defaultHandler = new DefaultHandler(); HandlerList handlers = new HandlerList(); handlers.setHandlers(new Handler[] {servletHandler,resourceHandler,defaultHandler}); server.setHandler(handlers); server.start(); server.join(); } }The server can be run from an IDE or via maven using the following command line:
mvn mvn -Pserver exec:exec
The Browser Client
The HTML for the chat room simply imports some CSS and the javascript before creating a few simple divs to contain the chat text, the join dialog and the joined dialog:
<html> <head> <title>WebSocket Chat Example</title> <script type='text/javascript' src="chat.js"></script> <link rel="stylesheet" type="text/css" href="chat.css" /> </head> <body> <div id='chat'></div> <div id='input'> <div id='join' > Username: <input id='username' type='text'/> <input id='joinB' class='button' type='submit' name='join' value='Join'/> </div> <div id='joined' class='hidden'> Chat: <input id='phrase' type='text'/> <input id='sendB' class='button' type='submit' name='join' value='Send'/> </div> </div> <script type='text/javascript'>init();</script> </body> </html>The javascript create a room object with methods to handle the various operations of a chat room. The first operation is to join the chat room, which is triggered by entering a user name. This creates a new WebSocket object pointing to the /chat URL path on the same server the HTML was loaded from:
var room = { join : function(name) { this._username = name; var location = document.location.toString() .replace('http://', 'ws://') .replace('https://', 'wss://')+ "chat"; this._ws = new WebSocket(location, "chat"); this._ws.onopen = this.onopen; this._ws.onmessage = this.onmessage; this._ws.onclose = this.onclose; }, onopen : function() { $('join').className = 'hidden'; $('joined').className = ''; $('phrase').focus(); room.send(room._username, 'has joined!'); }, ...The javascript websocket object is initialised with call backs for onopen, onclose and onmessage. The onopen callback is handled above by switching the join div to the joined div and sending a “has joined” message.
Sending is implemented by creating a string of username:message and sending that via the WebSocket instance:
... send : function(user, message) { user = user.replace(':', '_'); if (this._ws) this._ws.send(user + ':' + message); }, ...If the chat room receives a message, the onmessage callback is called, which sanitises the message, parses out the username and appends the text to the chat div:
... onmessage : function(m) { if (m.data) { var c = m.data.indexOf(':'); var from = m.data.substring(0, c) .replace('<','<') Â .replace('>','>'); var text = m.data.substring(c + 1) .replace('<', '<') Â .replace('>', '>'); var chat = $('chat'); Â var spanFrom = document.createElement('span'); spanFrom.className = 'from'; spanFrom.innerHTML = from + ':Â '; var spanText = document.createElement('span'); spanText.className = 'text'; spanText.innerHTML = text; var lineBreak = document.createElement('br'); chat.appendChild(spanFrom); chat.appendChild(spanText); chat.appendChild(lineBreak); chat.scrollTop = chat.scrollHeight - chat.clientHeight; } }, ...Finally, the onclose handling empties the chat div and switches back to the join div so that a new username may be entered:
... onclose : function(m) { this._ws = null; $('join').className = ''; $('joined').className = 'hidden'; $('username').focus(); $('chat').innerHTML = ''; } };With this simple client being served from the server, you can now point your websocket capable browsers at http://localhost:8080 and interact with the chat room. Of course this example glosses over a lot of detail and complications a real chat application would need, so I suggest you read my blog is websocket chat simpler to learn what else needs to be handled.
The Load Test Client
The jetty websocket java client is an excellent tool for both functional and load testing of a websocket based service. It uses the same endpoint API as the server side and for this example we create a simple implementation of the OnTextMessage interface that keeps track of the all the open connection and counts the number of messages sent and received:
public class ChatLoadClient implements WebSocket.OnTextMessage { private static final AtomicLong sent = new AtomicLong(0); private static final AtomicLong received = new AtomicLong(0); private static final Set<ChatLoadClient> members = new CopyOnWriteArraySet<ChatLoadClient>(); private final String name; private final Connection connection; public ChatLoadClient(String username,WebSocketClient client,String host, int port) throws Exception { name=username; connection=client.open(new URI("ws://"+host+":"+port+"/chat"),this).get(); } public void send(String message) throws IOException { connection.sendMessage(name+":"+message); } public void onOpen(Connection connection) { members.add(this); } public void onClose(int closeCode, String message) { members.remove(this); } public void onMessage(String data) { received.incrementAndGet(); } public void disconnect() throws IOException { connection.disconnect(); }The Websocket is initialized by calling open on the WebSocketClient instance passed to the constructor. The WebSocketClient instance is shared by multiple connections and contains the thread pool and other common resources for the client.
This load test example comes with a main method that creates a WebSocketClient from command line options and then creates a number of ChatLoadClient instances:
public static void main(String... arg) throws Exception { String host=arg.length>0?arg[0]:"localhost"; int port=arg.length>1?Integer.parseInt(arg[1]):8080; int clients=arg.length>2?Integer.parseInt(arg[2]):1000; int mesgs=arg.length>3?Integer.parseInt(arg[3]):1000; WebSocketClient client = new WebSocketClient(); client.setBufferSize(4096); client.setMaxIdleTime(30000); client.setProtocol("chat"); client.start(); Â // Create client serially ChatLoadClient[] chat = new ChatLoadClient[clients]; for (int i=0;i<chat.length;i++) chat[i]=new ChatLoadClient("user"+i,client,host,port); ...Once the connections are opened, the main method loops around picking a random client to speak in the chat room
... // Send messages Random random = new Random(); for (int i=0;i<mesgs;i++) { ChatLoadClient c = chat[random.nextInt(chat.length)]; String msg = "Hello random "+random.nextLong(); c.send(msg); } ...Once all the messages have been sent and all the replies have been received, the connections are closed:
... // close all connections for (int i=0;i<chat.length;i++) chat[i].disconnect();The project is setup so that the load client can be run with the following maven command:
mvn -Pclient exec:exec
And the resulting output should look something like:
Opened 1000 of 1000 connections to localhost:8080 in 1109ms Sent/Received 10000/10000000 messages in 15394ms: 649603msg/s Closed 1000 connections to localhost:8080 in 45ms
Yes that is 649603 messages per second!!!!!!!!!!! This is a pretty simple easy test, but it is still scheduling 1000 local sockets plus generating and parsing all the websocket frames. Real applications on real networks are unlikely to achieve close to this level, but the indications are good for the capability of high throughput and stand by for more rigorous bench marks shortly.