Webtide

Category: Uncategorized

  • Webtide Scaling Highlighted on InfoQ

    Based on Greg’s earlier benchmarking on Amazon EC2 nodes. Good stuff! It is great to demonstrate that open approaches can yield the best performance.

    InfoQ Link

  • Jetty and Webtide at Zimbra / Yahoo

    A great endorsement of the performance of Jetty for Ajax apps. (and of course, for the help Webtide can provide).

    Application Development Trends article link

    Zimbra’s blog entry on the subject.

    Zimbra serves millions of users, and we’re proud to work with them. Webtide’s services and support help many companies scale their apps rapidly and economically. Drop us a line if you’re interested to discuss.

  • Maven Archetypes Updated

    The maven webapp archetypes – all with configuration for the handy Jetty maven plugin built in –  made available by Webtide  have recently been updated to the following versions:

    • Active MQ – 5.0.0
    • Dojo – 1.0.2
    • DWR – 2.0.2
    • JSF – 1.2_07
    • Spring – 2.5
    • Struts – 2.0.11
    • Tapestry – 4.1.3
    • WebWork – 2.2.6

    We’ve also added a new archetype:

    • cometd – Bayeux/jetty 6.1.7

    The cometd archetype will generate a complete webapp with working end-to-end bayeux  ajax messaging.

    All are freely available for download from the Webtide site at www.webtide.com/resources.jsp.

  • Client-side session replication for clusters

    Web server clusters are used for scalability (handling more requests), availability (always have a server available) and fault tolerance (handling the failure of a server).  Of these, the fault tolerance is the hardest and most costly mechanism to implement as it requires that all session data be replicated off the web server so that it will be available to another web server if the initial webserver becomes unavailable. 
    Techniques for  replicating session data include distributing: to all nodes in a cluster; to a subset of the nodes in the cluster; to a database; to a cluster server; etc. etc. To this range of solutions, cometd/bayeux now adds the possibility of using the client machine to store replicated session.
    While it sounds strange, if you think about it, the client machine is the perfect place to store a replicated copy of the server session data.  It is a machine that is always in communication to the correct  webserver,   if the client machine crashes, the session is by definition over so you don’t need to timeout the replicated data. Clients machines scale with the number of users, so if you have more users then you have then more machines available to store replicated session.
    To test this concept I have created the BayeuxSessionManager for Jetty. This session manager keeps a dirty bit on the session, and if at the end of a request the session has been modified, then the session is serialized (should be encrypted) and B64 encoded and sent to the client as a Bayeux message.  The client saves the opaque blob of session data. If the client is directed to a new server (due to policy, maintenance, failure or happenstance), then the Bayeux protocol will see that the client ID is not know and a re-handshake will be invoked.  The session extension includes any saved blobs of session data with the handshake request, so that as the Bayeux connection is established with the new server, the session is restored and is again available.
    Now I don’t think this approach is ever going to be 100% and I certainly would not like to see any pacemakers using this to communicate with a medical database about how many volts to zap you with to save you from a heart attack.   But then it may be good enough to deal with enough common failures that you will be able to rest easy and avoid the stress levels the could cause a heart attack in the first place!
    At this stage, this is only a proof of concept and  more details do need to be  worked out, including:

    • encryption of the session data blob so that clients cannot see inside server data structures.
    • how to deal with page reloads: should the client use some client side persistence (or is it good enough to expect that page reloads are unlikely to coincide with server failures?)
    • how to deal with requests that need sessions that arrive on the new server before the bayeux connection is established.

    I think all of these are solvable and for applications that are clustered primarily for scalability and availability, this technique could provide a bit of fault tolerance without the need for complex server interconnections or repositories than can hold all sessions from all nodes.
    What this technique needs now is a real use-case to help drive it towards a complete solutions. So if anybody is interested in this style of replication and has a project that can afford to experiment with a new and novel approach, please contact me and we can see where this leads.

  • 20,000 reasons why Jetty scales

    I have published a comet Daily article that describes how I have benchmarked Jetty’s Cometd implementationof dojox Bayeux protocol to 20,000 simultaneous clients.  This blog looks at the same results from a more Jetty centric view point.


    The diagram above shows results of the benchmarking. These results are described in more detail in the article, but in summary they show that sub-second latency can be achieved even for 20,000 simultaneous clients – albeit with some reduced message throughput.
    Things to note with regards to Jetty are:

    • 20,000 simultaneous Bayeux connections means 20,000 simultaneous TCP/IP connections, 20,000 outstanding HTTP requests and 20,000  Continuations. This are good numbers for web-1.0 applications, let alone web-2.0 Ajax push applications.
    • To test this, I used the Jetty asynchronous HTTP client, which was able to also scale to 20,000 connections – a pretty amazing number!  Over 10,000 the client was a little brittle but with a bit more work chasing lock starvation this should be able to be resolved.
    • It was not all roses.  Jetty itself proved a little brittle over 10,000 connections until I replaced the ThreadPool.  The problem was lock starvation, as the threadpool only had a single lock that was just too much in demand.  I have written a new QueuedThreadPool that can be dropped into exist Jetty deployments that has 4 locks and a streamlined implementations.  It greatly improved the scalability. I will soon blog in detail how this can be applied.
    • 20,000 – I just like saying that number.
    • The default jetty.xml has a lowResources limit set at 5000 connections.  This is 15,000 to low when you can handle 20,000!
    • There are some more performance/scalability improvements in the pipeline with Jetty 6.2(trunk).

  • Blocking Servlets, Asynchronous Transport

    The web-2.0 Ajax Comet use-cases have been getting most the attention when it comes to asynchronous features of servlet containers.   The Asynchronous features are needed so that a request may wait without a thread allocated while waiting for a comet event to deliver to the Ajax client.

    However, there are some compelling use-cases for traditional web-1.0 applications to benefit greatly from asynchronous features.  Previously I have looked at how threadless waiting for resources can improve quality of service.  In this blog, I look at how an asynchronous servlet container like Jetty can greatly improve the scalability for serving medium and large content.  Furthermore, I reveal how your benchmarking is probably not giving you the full story when it comes to scalability.

    When applications are benchmarked and/or stress tested, it is frequently the case that while the server environment is a good approximation of the deployment environment, the simulated client environment is by necessity only a very rough approximation of the real world.   Consider the network speed between test client and test server, this is frequently a 100Mbs (or even a gigabit) LAN connection.    Unfortunately the load profile of 1000 simulated users share 1 fast connection can be vastly different from that of 1000 real users on 1000 ISP limited connections.

    Assuming all users are broadband and that average transfer rates can be maintained at an average of 1Mbs.   This means that content can take over 100 times as long to transfer over an ISP limited connection than over a fast local LAN.  This means that a 20KB logo takes 0.15s to serve, the 250KB content of a typical  home page  takes  1.9s, a  1MB  images takes 8s and a  5MB  mpeg or  pdf  takes  40s

    The problem with this latency, is that blocking servlet containers will need to allocate a thread while that content is being flushed to the client. If a blocking container is serving 10 mpegs per second then 400 threads could be needed by a non-asynchronous server just to flush the content! Serving 20KB dynamic content to 1000 requests per second needs 156 threads dedicated to flushing content above and beyond the threads needed to receive requests, handling them and generate the responses.

    While servlets are blocking, Jetty 6 has asynchronous features that allow threads to be freed while content is being flushed after a servlet has returned from service.   If the content being served is static content, then an NIO memory mapped buffer is used to directly send the content to the Socket Channel, a thread is only allocated to handle the initial request and the remainder of the content is flushed by the JVM and operating system without any user threads being allocated.  

    For dynamic content, Jetty 6 will buffer content until the buffer is filled, the servlet flushes the buffer or the servlet returns from the service method.  In this later case, Jetty is able to flush the buffer asynchronously and only allocate a thread for the short period needed to initiate the write of  each block of  data.  Because of Jetty’s flexible buffer  allocation mechanism, large  content buffers can be allocated which will mean that servlets will frequently run to completion without blocking and then the generated content is asynchronously flushed.

    For example, I ran Jetty 6  on a mid-sized amazon EC2 node serving 20KB dynamic content at 80 requests per second.    Over an 8Mbs link with a 180ms ping time over 40 threads were needed to when running in blocking mode (content flushed by servlet). In asynchronous mode, the content was served with only 2 threads!   When the same test was run locally, 2 threads were sufficient  in both cases.
     
    So in summary, there are two morals to this blog!  Firstly asynchronous features can have a significant impact on the scalability of your web application, even if you are not using any Ajax clients. Secondly, running benchmarks on fast networks may give you a very false reading on the true scalability of your application.

  • Zimbra, One of the Largest Collaboration Environments

    This one’s purely in their words… “In summary, we chose Jetty not only because it supports Comet in a scalable manner, but also because the Continuation implementation of Comet is least disruptive to existing Servlet based technologies.” -JJ Zhuang

    See their full posting with Webtide endorsement here!

  • Polar Rose

    Making a name for online image search

    Polar Rose’s unique technology gives greater meaning and context to digital photos by allowing them to be indexed online just like text documents. The company developer user friendly, fun, useful, transparent applications that have evolved from computer vision research at the Universities of Lund and Malmo in Sweden. Polar Rose is an active member of the open source community.

    Challenges: Polar Rose delivers a Firefox plug-in capable of scanning and passing photographic information to back-end servers. A vibrant community of users tag names on these photos and are the driving force behind identification.

    In order to fuel both community growth and indexing capacities, Polar Rose required a scalable two-way dynamic communication solution. To meet its exact needs, it also needed expert advice on implementation and customization.

    Solution: Polar Rose now uses the Hightide open source distribution of Jetty from Webtide on its servers. By using the Bayeux protocol for two-way communication implemented in the Cometd project together with Java by Webtide, developers can now use efficient and secure channel-based message system.

    Issues such as browser connection limits and sharing are transparently handled by Cometd. Jetty scales to host the communications using its asynchronous capabilities to optimally manage system resources. Add Webtide’s Developer Advice and Production Support and Polar Rose now has an economical, small footprint, high performance, open source infrastructure.


    Why Polar Rose Chose Webtide:     “Hightide currently provides the industry’s best Cometd support, which Polar Rose uses to get real-time feedback in its Firefox extension. Hightide also integrates many standard Java technologies in a simple lean and mean package which has removed the need for the more traditional fat application servers that one would otherwise use. Because it integrates so well in the development process and tools, it has allowed us to be significantly more agile in implementing new functionality to our rather complex solution.” -Stefan Arentz, System Architect, Polar Rose.

  • Photacular

    Photacular enables users to create fun, unique products from their photographs with safe, secure, 24/7 online access. Options include T-shirts, mugs, coasters, jewelry, holiday ornaments, canvas prints, and more. All are custom made.

    Challenges: The company sought to launch an easy to use rich web-based application for customizing photo-based items. The infrastructure had to be simple to deploy and highly scalable on commodity hardware in order to cost-effectively serve its customers with fast, reliable access to their products. Photacular needed access to the source code for possible customization and inspection. It also had to be backed by solid and easy-to-access support that would be available 24/7, just as they are.

    Solution: Photacular found that combination in Jetty with Webtide Developer Advice. In a period of weeks, Photacular went from concept to deployment, rapidly clearing any questions or obstacles that were in the way. The service is extremely easy to use with dynamic graphics and interactive features. Administration is straight forward as is deployment to hosting services. Its site performance is outstanding, and it is now open for business.

    “Working with Webtide and their developer advice system meant that we basically had them as a team member on call. Photacular could focus on its photo and product expertise and we could tap Webtide anytime for their server and Ajax knowledge. I am glad I went with Jetty as my app server.” -Darla Weiss, Founder, Photacular

    Please see Photacular.com to see it in action.

  • Hightide 6.1H.6 Release

    Hightide 6.1H.6 is available for download from the Webtide website here:
    http://www.webtide.com/downloads.jsp

    The changes in this release include:

    • jetty 6.1.6
    • atomikos version updated to 3.2.1
    • dwr updated to 2.0.2
    • debian packaging

    The Hightide wiki has more information, along with instructions on how to install the debian packages.