Category: HTTP

Introduction

One of the big refactorings in Jetty 9 is the complete rewrite of the HTTP client.
The reasons behind the rewrite are many:

• We wrote the codebase several years ago; while we have actively maintained, it was starting to show its age.
• The HTTP client guarded internal data structures from multithreaded access using the synchronized keyword, rather than using non-blocking data structures.
• We exposed as main concept the HTTP exchange that, while representing correctly what an HTTP request/response cycle is,  did not match user expectations of a request and a response.
• HTTP client did not have out of the box features such as authentication, redirect and cookie support.
• Users somehow perceived the Jetty HTTP client as cumbersome to program.

The rewrite takes into account many community inputs, requires JDK 7 to take advantage of the latest programming features, and is forward-looking because the new API is JDK 8 Lambda-ready (that is, you can use Jetty 9’s HTTP client with JDK 7 without Lambda, but if you use it in JDK 8 you can use lambda expressions to specify callbacks; see examples below).

Programming with Jetty 9’s HTTP Client

The main class is named, as in Jetty 7 and Jetty 8, org.eclipse.jetty.client.HttpClient (although it is not backward compatible with the same class in Jetty 7 and Jetty 8).
You can think of an HttpClient instance as a browser instance.
Like a browser, it can make requests to different domains, it manages redirects, cookies and authentications, you can configure it with a proxy, and it provides you with the responses to the requests you make.
You need to configure an HttpClient instance and then start it:

HttpClient httpClient = new HttpClient();
// Configure HttpClient here
httpClient.start();

Simple GET requests require just  one line:

ContentResponse response = httpClient
        .GET("http://domain.com/path?query")
        .get();

Method HttpClient.GET(...) returns a Future<ContentResponse> that you can use to cancel the request or to impose a total timeout for the request/response conversation.
Class ContentResponse represents a response with content; the content is limited by default to 2 MiB, but you can configure it to be larger.
Simple POST requests also require just one line:

ContentResponse response = httpClient
        .POST("http://domain.com/entity/1")
        .param("p", "value")
        .send()
        .get(5, TimeUnit.SECONDS);

Jetty 9’s HttpClient automatically follows redirects, so automatically handles the typical web pattern POST/Redirect/GET, and the response object contains the content of the response of the GET request. Following redirects is a feature that you can enable/disable on a per-request basis or globally.
File uploads also require one line, and make use of JDK 7’s java.nio.file classes:

ContentResponse response = httpClient
        .newRequest("http://domain.com/entity/1")
        .file(Paths.get("file_to_upload.txt"))
        .send()
        .get(5, TimeUnit.SECONDS);

Asynchronous Programming

So far we have shown how to use HttpClient in a blocking style, that is the thread that issues the request blocks until the request/response conversation is complete. However, to unleash the full power of Jetty 9’s HttpClient you should look at its non-blocking (asynchronous) features.
Jetty 9’s HttpClient fully supports the asynchronous programming style. You can write a simple GET request in this way:

httpClient.newRequest("http://domain.com/path")
        .send(new Response.CompleteListener()
        {
            @Override
            public void onComplete(Result result)
            {
                // Your logic here
            }
        });

Method send(Response.CompleteListener) returns void and does not block; the Listener provided as a parameter is notified when the request/response conversation is complete, and the Result parameter  allows you to access the response object.
You can write the same code using JDK 8’s lambda expressions:

httpClient.newRequest("http://domain.com/path")
        .send((result) -> { /* Your logic here */ });

HttpClient uses Listeners extensively to provide hooks for all possible request and response events, and with JDK 8’s lambda expressions they’re even more fun to use:

httpClient.newRequest("http://domain.com/path")
        // Add request hooks
        .onRequestQueued((request) -> { ... })
        .onRequestBegin((request) -> { ... })
        // More request hooks available
        // Add response hooks
        .onResponseBegin((response) -> { ... })
        .onResponseHeaders((response) -> { ... })
        .onResponseContent((response, buffer) -> { ... })
        // More response hooks available
        .send((result) -> { ... });

This makes Jetty 9’s HttpClient suitable for HTTP load testing because, for example, you can accurately time every step of the request/response conversation (thus knowing where the request/response time is really spent).

Content Handling

Jetty 9’s HTTP client provides a number of utility classes off the shelf to handle request content and response content.
You can provide request content as String, byte[], ByteBuffer, java.nio.file.Path, InputStream, and provide your own implementation of ContentProvider. Here’s an example that provides the request content using an InputStream:

httpClient.newRequest("http://domain.com/path")
        .content(new InputStreamContentProvider(
            getClass().getResourceAsStream("R.properties")))
        .send((result) -> { ... });

HttpClient can handle Response content in different ways:
The most common is via blocking calls that return a ContentResponse, as shown above.
When using non-blocking calls, you can use a BufferingResponseListener in this way:

httpClient.newRequest("http://domain.com/path")
        // Buffer response content up to 8 MiB
        .send(new BufferingResponseListener(8 * 1024 * 1024)
        {
            @Override
            public void onComplete(Result result)
            {
                if (!result.isFailed())
                {
                    byte[] responseContent = getContent();
                    // Your logic here
                }
            }
        });

To be efficient and avoid copying to a buffer the response content, you can use a Response.ContentListener, or a subclass:

ContentResponse response = httpClient
        .newRequest("http://domain.com/path")
        .send(new Response.Listener.Empty()
        {
            @Override
            public void onContent(Response r, ByteBuffer b)
            {
                // Your logic here
            }
        });

To stream the response content, you can use InputStreamResponseListener in this way:

InputStreamResponseListener listener =
        new InputStreamResponseListener();
httpClient.newRequest("http://domain.com/path")
        .send(listener);
// Wait for the response headers to arrive
Response response = listener.get(5, TimeUnit.SECONDS);
// Look at the response
if (response.getStatus() == 200)
{
    InputStream stream = listener.getInputStream();
    // Your logic here
}

Cookies Support

HttpClient stores and accesses HTTP cookies through a CookieStore:

Destination d = httpClient
        .getDestination("http", "domain.com", 80);
CookieStore c = httpClient.getCookieStore();
List cookies = c.findCookies(d, "/path");

You can add cookies that you want to send along with your requests (if they match the domain and path and are not expired), and responses containing cookies automatically populate the cookie store, so that you can query it to find the cookies you are expecting with your responses.

Authentication Support

HttpClient suports HTTP Basic and Digest authentications, and other mechanisms are pluggable.
You can configure authentication credentials in the HTTP client instance as follows:

String uri = "http://domain.com/secure";
String realm = "MyRealm";
String u = "username";
String p = "password";
// Add authentication credentials
AuthenticationStore a = httpClient.getAuthenticationStore();
a.addAuthentication(
    new BasicAuthentication(uri, realm, u, p));
ContentResponse response = httpClient
        .newRequest(uri)
        .send()
        .get(5, TimeUnit.SECONDS);

HttpClient tests authentication credentials against the challenge(s) the server issues, and if they match it automatically sends the right authentication headers to the server for authentication. If the authentication is successful, it caches the result and reuses it for subsequent requests for the same domain and matching URIs.

Proxy Support

You can also configure HttpClient  with a proxy:

httpClient.setProxyConfiguration(
    new ProxyConfiguration("proxyHost", proxyPort);
ContentResponse response = httpClient
        .newRequest(uri)
        .send()
        .get(5, TimeUnit.SECONDS);

Configured in this way, HttpClient makes requests to the proxy (for plain-text HTTP requests) or establishes a tunnel via HTTP CONNECT (for encrypted HTTPS requests).

Conclusions

The new Jetty 9  HTTP client is easier to use, has more features and it’s faster and better than Jetty 7’s or Jetty 8’s.
The Jetty project continues to lead the way when it’s about the Web: years ago with Jetty Continuations, then with Jetty WebSocket, recently with Jetty SPDY and now with the first complete, ready to use, JDK 8’s Lambda -ready HTTP client.
Go get it while it’s hot !
Maven coordinates:

    org.eclipse.jetty
    jetty-client
    9.0.0.M3

Direct Downloads:
Main jar: jetty-client.jar
Dependencies: jetty-http.jar, jetty-io.jar, jetty-util.jar