Where are all the JVM implementations?

There is a glaring scarcity of JVM (Java Virtual Machine) implementations out there, and that’s quite paradoxical: One one hand the number of languages targeting the  JVM is exploding and on the other hand the number of JVM implementations is shrinking. Actually the most ambitious virtual machine implementation of late takes great care in avoiding the word “Java”! I am referring here to Dalvik, Google’s own vm that ships with Android. The wikipedia entry, under the section Architecture goes to great length to explain why this is not a JVM: “Unlike Java VMs, which are stack machines, the Dalvik VM is a register-based architecture.” They start the description of the Architecture by saying what the vm is not…

First let me quickly go over the opening statement:  The number of JVM languages is not only growing but the adoption of these languages is growing as well. The latest TIOBE Index attests to this with 4 languages in the top 50: Java (1), Python(5- I know, technically it’s not for the JVM, it runs on the JVM), Groovy(36), Scala(50) and let’s not forget Clojure, JRuby, Rhino, etc… The number of popular JVMs is dwindling: Kaffe has not released any code in 3 years, JRockit will probably be discontinued by Oracle, Microsoft discontinued its JVM in 2009 (a bad thing?), Blackdown Java did not make it past JDK 1.4, and the list goes on and on.

How can we explain this paradox and what does this means to people making strategic decisions about platform adoption? It is obvious that the HotSpot JVM is geared more than ever towards multi-language support, I think that one of the most convincing presentations was given by John Rose at JavaOne 2010 in a talk entitled One VM Many Languages. Take a look at slide #9 detailing all the JVM optimizations that can potentially benefit all the languages by simplifying the static compiler (which now “just” needs to create .class files) while the HotSpot optimizations can take place at run-time. The work involved in creating such a JVM is obviously daunting but the pool of  motivated developers should also increase: More languages and systems than ever before depend on the success of the JVM. The recent flurry of lawsuits by Oracle against Google is certainly not helping. Nor is the recent announcement by IBM that they will turn their attention to OpenJDK: OpenJDK is an SDK implementation competing with Apache Harmony’s SDK, not a JVM implementation per se, but in the spirit of dwindling implementations that’s one less JDK implementation…

People making strategic decisions about platform adoption have the right to worry, just as Java the language got us used to one spec/many implementations, the specification of the Java Virtual Machine should have led to a healthy number of viable  proprietary and open-source implementations. At the moment these premises are not being fulfilled but there is still hope. I think that a major player should step in, overlook the very short-term prospects for financial gains and sponsor a decent open-source JVM effort that would dare say its name: Java Virtual Machine implementation. IBM (J9), are you listening?

Clients like REST part 3

This post is the third one of a mini-series:

Clients Like REST part 1	Apache HTTP Commons Client
Clients Like REST part 2	Native J2SE API Client
Clients Like REST part 3	JAX-WS client API

I will now show you how to access REST services using the JAX-WS client API, but first let me explain the motivation behind this selection. After all JAX-WS is typically used with SOAP services and assumes that a WSDL resides on the server side, so why bother at all with JAX-WS when you can  use the JAX-RS client API to consume logically enough REST services? I think that there are quite a few situations where JAX-WS makes sense, and besides JAX-RS examples to access a REST service abound, so here is a list of possible reasons for wanting to use JAX-WS to access a REST service:

• You want to use a single API on the client side to handle both SOAP and REST services (The Dispatch API is particularly flexible, allowing you to handle SOAP-style messages with Header + Payload or REST-style messages with just a Payload, you are also free to use a data binding tool other than JAXB if needs be, Castor comes to mind)
• You want to use an API included in the JDK
• You want to support synchronous, asynchronous and one-way calls without resorting to 3rd-party solutions like COMET
• You want to abstract the actual binding of the transport (JAX-WS allows you to use SOAP 1.1 over HTTP, SOAP 1.2 over HTTP or XML over HTTP)
• You want to abstract the class of objects used for messages: The JAX-WS Dispatch API mandates the support of at least three types (javax.xml.transform.Source, javax.xml.soap.SOAPMessage and javax.activation.DataSource useful for MIME messages)

We will go over the client code to access the same service – defined through the interface IAutoStatService – and I will note along the way where does the API feel awkward to work with and where it does have some very interesting potential. Again I will illustrate how to add/delete/get/get all and update AutoStatistics, but you will notice right away that everything is very much XML-oriented (as opposed to, say, json). The reason is simple: No matter what binding you use for transport (SOAP 1.1 over HTTP, SOAP 1.2 over HTTP or XML over HTTP) it assumes an XML format. This can be limiting, but unless you use a 3rd party client library such as Jettison you are stuck with XML manipulation when using the plain-vanilla JAX-WS API from the JDK. I made a deliberate attempt to avoid  using non-JDK API such as CXF on the client side even though classes such as JaxWsProxyFactoryBean can be quite useful. On the other hand if you are perfectly happy working with XML you will see that  JAX-WS integrates naturally with  the javax.xml.transform.Source and the javax.xml.transform.Transformer classes. From there any high-level XML API (I used XPath for e.g.) can be readily used.

On the awkward side you will note that JAX-WS expects a WSDL-oriented development and forces you to define two qualified names for the service and the port even though the Dispatch class allows you to handle raw XML. This is illustrated in the javax.xml.ws.Service.createDispatch(QName portName, Class<Source> type, Mode mode) method where you will set the Mode to Service.Mode.PAYLOAD as opposed to Service.Mode.MESSAGE to express the need to work with XML contents without a SOAP envelope, as is the case with REST services.

I also noticed a bug in the CXF implementation (2.2.3) I was using; even though my GET operations are invoked without an additional argument I could not simply call javax.xml.ws.Dispatch.invoke(null) as it would throw an exception, I had to pass in a fake request that would get ignored on the server side without any visible side-effect. My understanding is that the latest Glassfish and Axis2 do not suffer from such a bug. This shouldn’t detract you from using CXF on the server-side…

So here is the code:

package com.apptotest.client;

import static org.junit.Assert.*;
import java.io.ByteArrayInputStream;
import java.io.StringReader;
import java.io.StringWriter;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.xml.namespace.QName;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import javax.xml.ws.Dispatch;
import javax.xml.ws.Service;
import javax.xml.ws.handler.MessageContext;
import javax.xml.ws.http.HTTPBinding;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathFactory;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;

/**
 * We'll use the JAX-WS 2.x API (included in the SDK) to connect and test the following
 * REST operations:
 * - POST
 * - DELETE
 * - GET
 * - PUT
 */
public class AutoStatServiceImplJAXWSClientTest {
 private static final String hostname = "http://localhost:7650/autoStats/AutoStatService";
 private Transformer transformer;

 @Before
 public void setUp() throws Exception {
   transformer = TransformerFactory.newInstance().newTransformer();
   transformer.setOutputProperty(OutputKeys.INDENT, "no");
   transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
   transformer.setOutputProperty(OutputKeys.STANDALONE, "yes");
 }

 /**
 * corresponding to URL: http://localhost:7650/cxfweb_ajax/cxf/rest/CustomerService/add
 * @throws Exception
 */
 @Test
 public final void testAddAutoStat() throws Exception {
   String xmlAutoStatistics = "9000006150.0BMW 335ix13.3119.04.7";
   QName serviceName = new QName("http://test", "fakeservicename");
   QName portName = new QName("http://test", "fakeportname");
   Service service = Service.create(serviceName);
   service.addPort(portName, HTTPBinding.HTTP_BINDING, hostname + "/add");
   Dispatch dispatch = service.createDispatch(portName, Source.class, Service.Mode.PAYLOAD);
   Map requestContext = dispatch.getRequestContext();
   requestContext.put(MessageContext.HTTP_REQUEST_METHOD, "POST");
   Map> httpRequestHeaders = new HashMap>();
   httpRequestHeaders.put("Content-Type", Arrays.asList(new String[] { "application/xml" }));
   httpRequestHeaders.put("Accept", Arrays.asList(new String[] { "application/xml" }));
   httpRequestHeaders.put("Content-Length", Arrays.asList(new String[] { Integer.toString(xmlAutoStatistics.length()) }));
   requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);
   Source result = dispatch.invoke(new StreamSource(new ByteArrayInputStream(xmlAutoStatistics.getBytes())));
   StringWriter writer = new StringWriter();
   transformer.transform(result, new StreamResult(writer));

   Map responseContext = dispatch.getResponseContext();
   if (!responseContext.get(org.apache.cxf.message.Message.RESPONSE_CODE).equals(new Integer(200))) {
      fail("GET method failed: " + responseContext.get("org.apache.cxf.service.model.MessageInfo"));
   } else {
      String expectedResponse = "9000006150.0BMW 335ix13.3119.04.7";
     assertTrue(writer.toString().contains(expectedResponse));
   }
 }

 /**
 * corresponding to URL: http://localhost:7650/autoStats/AutoStatService/delete/{id}
 * where {id} gets expanded by the REST template
 * @throws Exception
 */
 @Test
 public final void testDeleteAutoStat() throws Exception {
   QName serviceName = new QName("http://test", "fakeservicename");
   QName portName = new QName("http://test", "fakeportname");
   Service service = Service.create(serviceName);
   service.addPort(portName, HTTPBinding.HTTP_BINDING, hostname + "/delete/10003");
   Dispatch dispatch = service.createDispatch(portName, Source.class, Service.Mode.PAYLOAD);
   Map requestContext = dispatch.getRequestContext();
   requestContext.put(MessageContext.HTTP_REQUEST_METHOD, "DELETE");
   Map> httpRequestHeaders = new HashMap>();
   httpRequestHeaders.put("Content-Type", Arrays.asList(new String[] { "application/xml" }));
   httpRequestHeaders.put("Accept", Arrays.asList(new String[] { "application/xml" }));
   requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);

   String sampleBodyContent = ""
   + "HELLO THERE!!!" + "";
   DOMSource request = createDOMSourceFromString(sampleBodyContent);
   Source result = dispatch.invoke(request);
   StringWriter writer = new StringWriter();
   transformer.transform(result, new StreamResult(writer));

   Map responseContext = dispatch.getResponseContext();
   if (!responseContext.get(org.apache.cxf.message.Message.RESPONSE_CODE).equals(new Integer(200))) {
   fail("GET method failed: " + responseContext.get("org.apache.cxf.service.model.MessageInfo"));
   } else {
      String expectedResponse = "";
      assertTrue(writer.toString().contains(expectedResponse));
   }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/all
 * @throws Exception
 */
 @Test
 public final void testGetAutoStats() throws Exception {
   QName serviceName = new QName("http://test", "fakeservicename");
   QName portName = new QName("http://test", "fakeportname");
   Service service = Service.create(serviceName);
   service.addPort(portName, HTTPBinding.HTTP_BINDING, hostname + "/all");
   Dispatch dispatch = service.createDispatch(portName, Source.class, Service.Mode.PAYLOAD);

   Map requestContext = dispatch.getRequestContext();
   requestContext.put(MessageContext.HTTP_REQUEST_METHOD, "GET");
   Map> httpRequestHeaders = new HashMap>();
   httpRequestHeaders.put("content-type", Arrays.asList(new String[]{ "application/xml;charset=UTF-8" }));
   httpRequestHeaders.put("Content-Type", Arrays.asList(new String[]{ "application/xml;charset=UTF-8" }));
   httpRequestHeaders.put("Accept", Arrays.asList(new String[]{ "application/xml" }));
   requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);

   String sampleBodyContent = ""
   + "HELLO THERE!!!" + "";
   DOMSource request = createDOMSourceFromString(sampleBodyContent);
   Source result = dispatch.invoke(request);
   StringWriter writer = new StringWriter();
   transformer.transform(result, new StreamResult(writer));

   XPath xpath= XPathFactory.newInstance().newXPath();
   InputSource xmlSource = new InputSource(new StringReader(writer.toString()));

   Map responseContext = dispatch.getResponseContext();
   if (!responseContext.get(org.apache.cxf.message.Message.RESPONSE_CODE).equals(new Integer(200))) {
     fail("GET method failed: " + responseContext.get("org.apache.cxf.service.model.MessageInfo"));
   } else {
      XPathExpression allAutoStatisticsExpr = xpath.compile("//AllAutoStatistics");
      Element allStatsElement = (Element) allAutoStatisticsExpr.evaluate(xmlSource, XPathConstants.NODE);
      assertNotNull(allStatsElement);
      NodeList allStatsChildren = allStatsElement.getChildNodes();
      assertTrue("AllAutoStatistics num,of children: " + allStatsChildren.getLength(), allStatsChildren.getLength() >= 6);
      Node allStatsChild = allStatsChildren.item(0);
      assertTrue(allStatsChild.hasChildNodes());
   }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/autostats/1000002
 * @throws Exception
 */
 @Test
 public final void testGetAutoStatAsXml() throws Exception {
   QName serviceName = new QName("http://test", "fakeservicename");
   QName portName = new QName("http://test", "fakeportname");
   Service service = Service.create(serviceName);
   service.addPort(portName, HTTPBinding.HTTP_BINDING, hostname + "/autostats/1000002");
   Dispatch dispatch = service.createDispatch(portName, Source.class, Service.Mode.PAYLOAD);

   Map requestContext = dispatch.getRequestContext();
   requestContext.put(MessageContext.HTTP_REQUEST_METHOD, "GET");
   Map> httpRequestHeaders = new HashMap>();
   httpRequestHeaders.put("content-type", Arrays.asList(new String[]{ "application/xml;charset=UTF-8" }));
   httpRequestHeaders.put("Content-Type", Arrays.asList(new String[]{ "application/xml;charset=UTF-8" }));
   httpRequestHeaders.put("Accept", Arrays.asList(new String[]{ "application/xml" }));
   requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);

   String sampleBodyContent = ""
   + "HELLO THERE!!!" + "";
   DOMSource request = createDOMSourceFromString(sampleBodyContent);

   Source result = dispatch.invoke(request);
   StringWriter writer = new StringWriter();
   transformer.transform(result, new StreamResult(writer));

   Map responseContext = dispatch.getResponseContext();
   if (!responseContext.get(org.apache.cxf.message.Message.RESPONSE_CODE).equals(new Integer(200))) {
      fail("GET method failed: " + responseContext.get("org.apache.cxf.service.model.MessageInfo"));
   } else {
      String expectedResponse = "1000002181.0Alfa Romeo 8C Competizione12.4105.04.2";
      assertTrue(writer.toString().contains(expectedResponse));
   }
 }

 /**
 * corresponding URL: http://localhost:7650/cxfweb_ajax/cxf/rest/CustomerService/edit
 * @throws Exception
 */
 @Test
 public final void testUpdateAutoStat() throws Exception {
   String xmlAutoStatistics = "1000003130.0Audi A5 2.0T Quattro - Updated JAX-WS14.8130.06.2";
   QName serviceName = new QName("http://test", "fakeservicename");
   QName portName = new QName("http://test", "fakeportname");
   Service service = Service.create(serviceName);
   service.addPort(portName, HTTPBinding.HTTP_BINDING, hostname + "/edit");
   Dispatch dispatch = service.createDispatch(portName, Source.class, Service.Mode.PAYLOAD);

   Map requestContext = dispatch.getRequestContext();
   requestContext.put(MessageContext.HTTP_REQUEST_METHOD, "POST");
   Map> httpRequestHeaders = new HashMap>();
   httpRequestHeaders.put("Content-Type", Arrays.asList(new String[] { "application/xml" }));
   httpRequestHeaders.put("Accept", Arrays.asList(new String[] { "application/xml" }));
   httpRequestHeaders.put("Content-Length", Arrays.asList(new String[] { Integer.toString(xmlAutoStatistics.length()) }));
   requestContext.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);

   Source result = dispatch.invoke(new StreamSource(new ByteArrayInputStream(xmlAutoStatistics.getBytes())));
   StringWriter writer = new StringWriter();
   transformer.transform(result, new StreamResult(writer));

   Map responseContext = dispatch.getResponseContext();
   if (!responseContext.get(org.apache.cxf.message.Message.RESPONSE_CODE).equals(new Integer(200))) {
      fail("GET method failed: " + responseContext.get("org.apache.cxf.service.model.MessageInfo"));
   } else {
      assertTrue(writer.toString().contains(expectedResponse));
   }
 }

 /**
 * @param input
 * @return
 * @throws Exception
 */
 private DOMSource createDOMSourceFromString(String input) throws Exception {
   byte[] bytes = input.getBytes();
   ByteArrayInputStream stream = new ByteArrayInputStream(bytes);
   DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
   domFactory.setNamespaceAware(false);
   domFactory.setFeature("http://xml.org/sax/features/namespaces", false);
   domFactory.setFeature("http://xml.org/sax/features/validation", false);
   domFactory.setFeature("http://apache.org/xml/features/validation/schema", false);
   DocumentBuilder domBuilder = domFactory.newDocumentBuilder();
   Document domTree = domBuilder.parse(stream);
   Node node = domTree.getDocumentElement();
   DOMSource domSource = new DOMSource(node);
   return domSource;
 }
}

Clients like REST part 2

This post is the second one of a mini-series:

Clients Like REST part 1	Apache HTTP Commons Client
Clients Like REST part 2	Native J2SE API Client
Clients Like REST part 3	JAX-WS client API

This post will show you how to access the same service – defined through the interface IAutoStatService – using the native J2SE API and in particular how to take full advantage of the HttpURLConnection class. I think that this is a pretty powerful concept: You can access a web service from a Java client without a 3rd party web-service library and without the need to write some type of parser.

Here are some notes about this client implementation:

1. It relies on the lowest common denominator to all java apps only, the  JDK
2. To be complete I will mention that I am using the Jackson JSON processor to construct JSON strings from a POJO and vice versa de-serializing a JSON string coming on an input stream into a POJO; it’s just a convenience tool that does not contradict the above point (using just the native J2SE API)
3. Again, there is nothing in this API that is REST-specific or even Web Services-specific; this point is worth stressing, by properly setting up the @javax.ws.rs.Path annotation on the server class the client will be able to call the addAutoStatistics() operation, for example, simply by constructing a POST request in the form http://<host>:<port>/autoStats/AutoStatService/add
4. Since we are mainly relying on the HttpURLConnection class you will be forced to think in terms of the details of the HTTP protocol: What to set for Content-Type, Accept or Content-Length request properties? How to decode the HTTP response code? etc… But it’s not that bad and as a bonus your code would be easily portable to another language since these constructs are pretty generic
5. You are not limited to a particular data format: if the server side produces JSON, you can de-serialize JSON into POJO, if the server side produces XML, you can de-serialize XML into POJO or use XQuery, etc…
6. The testHttpURLConnection(HttpURLConnection connection) method is completely optional but helpful when debugging. It’s mainly included to show you how the various parts of the HTTP message (header/body) are constructed in each of the following cases: POST, DELETE and GET

package com.apptotest.client;

import static org.junit.Assert.*;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.Permission;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.Map.Entry;
import org.codehaus.jackson.map.ObjectMapper;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

/**
 * TODO: We'll use a standard J2SE client (based on {@link HttpURLConnection} to connect
 * and test the following REST operations:
 * - POST
 * - DELETE
 * - GET
 * - PUT
 * This type of coding requires (some) help from the Jackson JSON library to interpret the
 * {@link InputStream}.
 */
public class AutoStatServiceImplJ2SEClientTest {
 private static final String hostname = "http://localhost:7650/autoStats/AutoStatService";
 private URL url;
 private HttpURLConnection conn;
 private InputStream in;
 private OutputStream out;
 private ObjectMapper mapper;

 @Before
 public void setUp() throws Exception {
     mapper = new ObjectMapper();
 }

 @After
 public void tearDown() throws Exception {
     if (in != null) { in.close(); }
     if (out != null) { out.close(); }
     if (conn != null) { conn.disconnect(); }
 }

 /**
 * corresponding to URL: http://localhost:7650/autoStats/AutoStatService/add
 * @throws Exception
 */
 @Test
 public final void testAddAutoStat() throws Exception {
     AutoStatistics stat = new AutoStatistics("BMW 335i", 9000007L, 150F, 4.7F, 13.3F, 119F);
     String jsonValue = mapper.writeValueAsString(stat);

     url = new URL(hostname + "/add");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("POST");
     conn.setDoOutput(true);
     conn.setUseCaches(false);
     conn.setRequestProperty("Content-Type", "application/json");
     conn.setRequestProperty("Accept", "application/json");
     conn.setRequestProperty("Content-Length", Integer.toString(jsonValue.length()));
     conn.getOutputStream().write(jsonValue.getBytes());
     conn.getOutputStream().flush();
     conn.connect();

     testHttpURLConnection(conn);

     if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
         fail("POST method failed: " + conn.getResponseCode() + "\t" + conn.getResponseMessage());
     } else {
         InputStream responseContent = (InputStream) conn.getContent();
         AutoStatistics respStat1 = mapper.readValue(responseContent, AutoStatistics.class);
         assertNotNull(respStat1);
         assertEquals(stat, respStat1);
     }
 }

 /**
 * corresponding to URL: http://localhost:7650/cxfweb_ajax/cxf/rest/AutoStatService/delete/{id}
 * where {id} gets expanded by the REST template
 */
 @Test
 public final void testDeleteAutoStat() throws Exception {
     url = new URL(hostname + "/delete/1000001");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("DELETE");
     conn.setDoOutput(true);
     conn.setUseCaches(false);
     conn.connect();

     testHttpURLConnection(conn);

     if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
         fail("DELETE method failed: " + conn.getResponseCode() + "\t" + conn.getResponseMessage());
     } else {
         InputStream responseContent = (InputStream) conn.getContent();
         AutoStatistics respStat1 = mapper.readValue(responseContent, AutoStatistics.class);
         assertNotNull(respStat1);
         assertEquals(new AutoStatistics(), respStat1);
     }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/all
 * @throws Exception
 */
 @Test
 public final void testGetAutoStats() throws Exception {
     url = new URL(hostname + "/all");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("GET");
     conn.connect();
     in = conn.getInputStream();

     testHttpURLConnection(conn);

     if (conn.getContentType().equals("application/json")) {
         mapper = new ObjectMapper();
         AllAutoStatistics stats = mapper.readValue(in, AllAutoStatistics.class);
         assertNotNull(stats);
         assertNotNull(stats.getAllStats());
         Collection<AutoStatistics> allStats = (Collection<AutoStatistics>) stats.getAllStats();
         assertTrue(allStats.contains(new AutoStatistics("Alfa Romeo 8C Competizione", 1000002L, 181F, 4.2F, 12.4F, 105F)));
         assertTrue(allStats.contains(new AutoStatistics("Cadillac CTS-V", 1000004L, 191F, 4.1F, 12.3F, 114F)));
     }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/autostats/1000002
 * @throws Exception
 */
 @Test
 public final void testGetAutoStatAsXml() throws Exception {
     url = new URL(hostname + "/autostats/1000002");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("GET");
     conn.setRequestProperty("Content-Type", "application/xml;charset=UTF-8");
     conn.setRequestProperty("Accept", "application/xml");
     conn.connect();
     in = conn.getInputStream();

     testHttpURLConnection(conn);

     if (conn.getContentType().equals("application/xml")) {
         char[] buff = new char[128];
         InputStreamReader isr = new InputStreamReader(in);
         StringBuilder builder = new StringBuilder();
         while (isr.read(buff) != -1) {
             builder.append(buff);
         }
         assertTrue(builder.length() > 0);
         String expectedResponse = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><AutoStatistics><id>1000002</id><maxSpeed>181.0</maxSpeed><name>Alfa Romeo 8C Competizione</name><quarterMileTimeInSecs>12.4</quarterMileTimeInSecs><sixtyToZeroDistanceInFt>105.0</sixtyToZeroDistanceInFt><zeroToSixtyTimeInSecs>4.2</zeroToSixtyTimeInSecs></AutoStatistics>";
         assertTrue(builder.toString().contains(expectedResponse));
     }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/autostats/1000002
 *
 * @throws Exception
 */
 @Test
 public final void testGetAutoStatAsJson() throws Exception {
     url = new URL(hostname + "/autostats/1000002");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("GET");
     conn.connect();
     in = conn.getInputStream();

     testHttpURLConnection(conn);

     if (conn.getContentType().equals("application/json")) {
         mapper = new ObjectMapper();
         AutoStatistics stat = mapper.readValue(in, AutoStatistics.class);
         assertNotNull(stat);
         assertEquals("name", "Alfa Romeo 8C Competizione", stat.getName());
         assertEquals("id", 1000002L, stat.getId().longValue());
         assertEquals("maxSpeed", 181.0F, stat.getMaxSpeed().floatValue(), 0.0);
         assertEquals("zeroToSixtyTimeInSecs", 4.2F, stat.getZeroToSixtyTimeInSecs().floatValue(), 0.0);
         assertEquals("quarterMileTimeInSecs", 12.4F, stat.getQuarterMileTimeInSecs().floatValue(), 0.0);
         assertEquals("sixtyToZeroDistanceInFt", 105.0F, stat.getSixtyToZeroDistanceInFt().floatValue(), 0.0);
     }
 }

 /**
 * corresponding URL: http://localhost:7650/autoStats/AutoStatService/edit
 * @throws Exception
 */
 @Test
 public final void testUpdateAutoStat() throws Exception {
     AutoStatistics stat = new AutoStatistics("Audi A5 2.0T Quattro - Updated J2SE", 1000003L, 130F, 6.2F, 14.8F, 130F);
     String jsonValue = mapper.writeValueAsString(stat);

     url = new URL(hostname + "/edit");
     conn = (HttpURLConnection) url.openConnection();
     conn.setRequestMethod("POST");
     conn.setDoOutput(true);
     conn.setUseCaches(false);
     conn.setRequestProperty("Content-Type", "application/json");
     conn.setRequestProperty("Accept", "application/json");
     conn.setRequestProperty("Content-Length", Integer.toString(jsonValue.length()));
     conn.getOutputStream().write(jsonValue.getBytes());
     conn.getOutputStream().flush();
     conn.connect();

     testHttpURLConnection(conn);

     if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
         fail("POST method failed: " + conn.getResponseCode() + "\t" + conn.getResponseMessage());
     } else {
         InputStream responseContent = (InputStream) conn.getContent();
         AutoStatistics respStat1 = mapper.readValue(responseContent, AutoStatistics.class);
         assertNotNull(respStat1);
         assertEquals(stat, respStat1);

         // and test
         url = new URL(hostname + "/all");
         conn = (HttpURLConnection) url.openConnection();
         conn.setRequestMethod("GET");
         conn.connect();
         in = conn.getInputStream();
         AllAutoStatistics stats = mapper.readValue(in, AllAutoStatistics.class);
         assertNotNull(stats);
         assertNotNull(stats.getAllStats());
         Collection<AutoStatistics> allStats = stats.getAllStats();
         assertTrue(allStats.contains(new AutoStatistics("Audi A5 2.0T Quattro - Updated J2SE", 1000003L, 130F, 6.2F, 14.8F, 130F)));
     }
 }

 /**
 * @param connection
 * @throws Exception
 */
 private void testHttpURLConnection(HttpURLConnection connection) throws Exception {
     boolean connAllowUserInteraction = connection.getAllowUserInteraction();
     String connContentType = connection.getContentType();
     String connContentEncoding = connection.getContentEncoding();
     String connRequestMethod = connection.getRequestMethod();
     boolean connDoInput = connection.getDoInput();
     boolean connDoOutput = connection.getDoOutput();
     Permission connPermission = connection.getPermission();
     URL connURL = connection.getURL();
     Map<String, List<String>> connHeaderFields = connection.getHeaderFields();

     System.out.println("connAllowUserInteraction: " + connAllowUserInteraction);
     System.out.println("connContentType: " + connContentType);
     System.out.println("connContentEncoding: " + connContentEncoding);
     System.out.println("connRequestMethod: " + connRequestMethod);
     System.out.println("connDoInput: " + connDoInput);
     System.out.println("connDoOutput: " + connDoOutput);
     System.out.println("connPermission: " + connPermission);
     System.out.println("connURL: " + connURL);

     if (connHeaderFields != null) {
         Set<Entry<String, List<String>>> connHeaderFieldsEntries = connHeaderFields.entrySet();
         for (Entry<String, List<String>> entry : connHeaderFieldsEntries) {
             System.out.println("connHeaderField: " + entry);
         }
     }
 }

}

JavaOne 2010 – Developing Composite Applications for the Cloud with Apache Tuscany (SCA)

A great take on a great technology, SCA (Service Component Architecture), was given by Jean-Sebastien Delfino (IBM) and Luciano Resende (Shutterfly). Can Tuscany (a SCA  implementation) shield you from the complexities of component assembly/component integration/deployment/inter-component communications/client protocols/… in the Cloud? It turns out that many of these problems are not specific to the Cloud, they apply to any distributed environment, but the live demo made it a point to show how SCA works in a Cloud.

First they quickly reviewed the SCA goals:

1. Abstract out the APIs, protocols and QoS (as in addressing, authentication levels, etc…) that define a service
2. A structure for application componentization (refer to my earlier post Patterns For Modularity )
3. A way to assemble/wire/re-wire components
4. Encourage Open Source implementations (Tuscany, Fabric3, etc…)
5. Encourage products implementations (IBM WebSphere Process Server, Oracle has an implementation inside its BPEL Process Manager, Tibco ActiveMatrix, SAP NetWeaver Platform, etc…)
6. Target SOA, multi-languages applications, application integration

But how can SCA help you in the Cloud? To answer this we need to look at the SCA Composite (briefly, in the SCA Assembly Model a Composite is the main unit of work  providing Components to implement the business function, exposing the Services  through various bindings such as WS or JMS, and listing its dependencies on other Composites through References). A Composite is defined in a SCDL (Service Component Definition Language) file. The Components can be coded in Java (POJOs or Spring Beans for e.g.), BPEL, JEE, Groovy, XQuery, etc… The multi-language support means that a Component can target different Cloud vendors’ run-times. The Service bindings are similarly diversified (WS, JMS, JCA, SLSB or Stateless Session Bean, HTTP, JSON, etc…) which means that various Cloud clients can access the same Component through various bindings. Finally one Composite can refer to another co-located or remotely located Composite, which makes them suitable for distributed environments such as the Cloud.

Let’s take a look at the Apache Tuscany project run-time. It comes in two flavors: Java, the most common run-time and the SCA native run-time (Components can be written in Python for potential deployments in the Google Application Engine Cloud or can be written in C++ – for situations requiring a small footprint). Scheme support for Components is currently experimental.

The demo was quite convincing: A SCA Composite was written in Java and deployed to Amazon EC2. Another Java Composite, part of the same application, was  pushed to Google Application Engine. That same Composite’s Component was re-written in Python and moved to Google Application Engine Cloud, the application behaved as before. Finally that same Composite’s Component was re-written in C++ and pushed to Amazon EC2, the application behaved as before. What’s worth mentioning is the minimal effort  required to move from one Component implementation to the other: Just one line of change in the SCDL .composite file.

So far so good but as mentioned before not really Cloud-specific. The Tuscany folks have recently forked another project, Nuvem, to address some Cloud-specific issues. Various utilities standardize access to Cloud services such as User Authentication and Authorization, Distributed Cache, Data Store and Queuing. This nicely complements the libcloud effort described in my earlier post where we saw common functionality to manage the various Clouds (list, reboot, create, destroy, etc…) getting standardized but Nuvem seems redundant with some areas of the Simple Cloud API (such as Queuing). Right now Nuvem is concentrating on Amazon and Google Application Engine.  What I liked about Nuvem was the REST-approach: REST itself is Cloud friendly, all operations are described in 4 simple verbs, this should help simplify the API. But no demo was given of the Nuvem capabilities, I guess the project is still too young. Jean Sebastien actually shared a wish list for Nuvem: XMPP (for EC2, it’s there for GAE), Maven, OpenID, hierarchical cache, key/value store, RDBMS, etc…

In conclusion developing a distributed application for deployment in your data center should not be fundamentally different from developing that same application for deployment in the Cloud. It’s the plumbing that differs but that’s transparent to the application. So the Cloud in itself should not be enough to entice you to adopt Tuscany or another SCA implementation. The complexity of the application, the desire to make it modular and service-oriented, the need to expose it business logic through many bindings and the option to support many component models (read many languages/frameworks) should entice you to adopt Tuscany or another SCA implementation.

JavaOne 2010 – Keeping your options open if the Cloud is not

This was one of the best presentations at JavaOne, probably due to the oratory talents of Doug Tidwell.

Doug presented libcloud and the Simple Cloud API, respectively a common library for interacting with the popular cloud server providers and controlling their VMs (reboot/create/destroy/list/images) and a  common interface for the three most common cloud application services (File Storage, Document Storage and Simple Queues). The whole idea is to code to those common APIs rather than using the vendor’s own API directly in an effort to increase portability (the ability to move a piece of code from one vendor API to the other) and interoperability (the ability to run the same piece of code across multiple vendors’ APIs). Common sense dictates that portability is harder to achieve for vendor APIs that provide the most functionality. Here is a list of some vendors for which the common interfaces were built:

• Amazon Relation Database Service (RDS)
• Amazon SimpleDB
• Amazon Machine Image (AMI) hosting an application container
• Microsoft Azure
• Google Application Engine

Before  proceeding I would like to mention another effort in this area; jclouds has an interesting API and Apache Nuvem (from the folks who brought you Tuscany) focuses more on the application layer but is at an early development stage.

Of course the community has been hotly debating the two initiatives (libcloud and Simple Cloud API) arguing that it may be too early to begin standardizing access to the Cloud for fear of locking their design while people are still trying to figure out the best way to accomplish some tasks. Doug, on the other hand, argues that we already know what needs to be done to manage a VM (start, stop, provision, etc…) and interoperability is the real issue that needs to be addressed so it’s not too early for some standard to emerge. I do think that the common interface approach makes sense, especially since we don’t know how many of those vendors will be around/bought/merged/etc… and some large systems will need to interoperate.

The path to libcloud was described as follows:

1. First we worried about APIs that deal with the XML and the JSON that goes into the Request and Response message i.e. the focus was on the wire
2. Then we considered language-specific features to handle SOAP or REST messages
3. Then we considered service-specific features while thinking about our organization needs and that’s the right level; the common interfaces should handle all the plumbing underneath the messages that get sent to the various Clouds

I will list here a purposefully boring-looking list of libcloud APIs to give a sense of 1) what the project tries to achieve (the methods are self-descriptive) and 2) the effort required to build such as an API since a deep dive with the various vendors APIs will show that not all capabilities are equally supported: Driver. getName(), Driver.listImages(), Driver.listLocations(), Driver.listNodes(), Driver.listSizes(), Driver.getId(), Driver.getPrivateId(), Driver.getPublicId(), Driver.getUuid() and Driver.getState().

The Simple Cloud API is a join effort between Zend, GoGrid, IBM, Microsoft, Nirvanix and Rackspace. Its makes heavy usage of Dependency Injection and configuration files to keep the code free of vendors’ specifics. For File Storage you would use a StorageAdapter to invoke common operations such as storeItem(), fetchItem(), deleteItem(), copyItem(), moveItem(), renameItem(), listFolders(), listItems(), storeMetadata(), fetchMetadata() and deleteMetadata(). However not all File Storage systems support, for example, renaming files, so proper exception flows must be put in place when using the API, and Doug pointed out that the best way for the common interface to handle these cases is still being debated: Introspection, instanceof, XSLT-style?

For Document Storage a common interface is trickier: Some data stores are relational, most aren’t, some support schema, most don’t, some support concurrency, most don’t, etc… The other issues arise from  the nature of Clouds’ Document Storage implementations: They are designed to scale to infinity and as such have no concept of indexed keys (you need to create one using Java Uuid, for example), they tend to be de-normalized (Amazon has no joins on tables), etc… I think that by Document Storage a paradigm shift is expected, indeed the name is so generic to indicate that it most probably does not refer to a relational database.

Finally Simple Queue: Right now it supports Amazon and Azure queues through a simple API: createQueue(), deleteQueue(), listQueues(), sendMessage(), receiveMessage(), deleteMessage(). SQS lets you peek at the Queue, Azure does not. Cloud queues are known to experience large delays and I think that there are probably two ways to look at it: A delay of around 30 seconds (between a Producer and a Consumer)  is so large that Cloud Queues are not worth looking at (un-reasonable) or a delay of around 30 seconds is quite large but then again Queue-based systems should be decoupled and Cloud Queues should not be used for low-latency situations (more reasonable).

Doug wrapped it with a demo where a Producer running an Order Processing system across many VMs on one Cloud created order details and placed them in a “Cloud DB”, created then an Order Message and placed it in a “Cloud Queue”, a second Order Processing running on another cloud picked the message from the “Cloud Queue”, got the order’s details from the “Cloud DB”, stored the invoice in a “Cloud Storage” and deleted the message from the “Cloud Queue”. As you guessed expressions between quotes refer to common interfaces and the two clouds were from different vendors. It worked like a charm and I will be eager to experiment with this setup.

In conclusion Cloud vendors are providing drastically different services so it makes sense to experiment with those two common interfaces; in particular libcloud should prove to be a productivity tool. Even the Simple Cloud API is an interesting abstraction layer that development teams would probably consider building in-house anyway if it did not exist, but I think that it will evolve quite a bit.

JavaOne 2010 – Effective XML: Leveraging JAXB and SDO

Blaise Doughan (Team lead for the TopLink and the EclipseLink JAXB & SDO projects) gave a very informative talk comparing and contrasting two mapping technologies: JAXB and SDO. By the way let me make it clear that mapping and binding are two distinct things: In this context, a mapping framework maps a Java Class to an XML Schema or an XML Document (if a schema is not available) and vice versa while a binding frameworks maintains a live connection between the Java Object and the XML document, and that’s a very powerful concept (I know it’s a simplistic defintion, Mark Hansen gives a more detailed description in his book SOA Using Java Web Services). JAXB is a mapping and a binding framework, SDO is a mapping framework.

Blaise quickly explained the mechanics behind JAXB and SDO binding mechanisms and pointed out that the most important thing, actually, is not their API for manipulating XML and bridging the gap between the two domains (Java and XML), rather it is their ability to play nicely with other frameworks to persist this data and represent/expose this data. In other words data binding is not an end in itself but a transitional (and extremely important) phase before data gets manipulated by the likes of JPA, JAX-WS, JAX-RS and SCA.

Now let’s look at how JAXB and SDO differ:

• At design time JAXB generates annotated POJO while SDO generates annotated/non-annotated Interface. Because JAXB generates plain POJOs serialization (say for RMI) and reuse are not an issue. SDO, on the other hand, uses a DataObject with a much richer meta model (Change Summary, Type and Sequence). To generate a GUI representation from a JAXB POJO you would need to use reflection while an SDO DataObject exposes directly attributes such as name, uri, baseType, dataType, open and sequenced
• The rich meta-data model of SDO is worth looking at since that’s were the designers spent most of their time: An SDO DataObject was designed to be language-agnostic and supports sophisticated object relationships at run-time: The open property, for example, refer to a set of attributes  that apply to certain instances of the Interface and not others (useful for versionning)
• Change Tracking: JAXB has none while SDO has a nice Change Summary to track created/modified/deleted DataObject(s), get the original values and undo changes. That’s all great but it duplicates many of the JPA  capabilities and when used with JPA it’s just overhead.
• Run-time model: The JAXB  model is quite simple, you deal with only a handful of objects: JAXBContext, Marshaller, Unmarshaller and the Binder (ref the earlier discussion on mapping v/s binding – briefly the Binder allows to preserve the Infoset). The SDO run-time is much more complex due to the fact that DataObject is a very generic entity. You start with a HelperContext and invoke on an as-needed basis an XSDHelper, an XMLHelper, a CopyHelper, a JavaHelper, an EqualityHelper, a TypeHelper, a DataFactory and a DataHelper. You get the picture…
• Switching between run-times: Now that’s an argument which really makes a difference. In JAXB you simply change one line in the jaxb.properties file! In SDO it’s not quite simple because no two implementations will generate the same interfaces with the same exact names, the specs are much looser. In my own experience I have switched during development between many different implementations because one would be more optimized for a flat schema, another would bet better at handlingh  large DOM and another would simply have bugs!
• XML-to-Java: JAXB has an edge because it was meant for Java while SDO can handle any Source

The session concluded with a case study: How to build a Data Access Service (DAS) that reads/writes from/to a data source and exposes the resulting objects through JSON or XML. JPA is used for persistence. The JAXB story is a straightforward one and you end up /start with properly annotated classes (@XmlRootElement, @Entity, etc…). The SDO story is not so simple since it was not meant to emit POJOs, so you could use JAXB to generate POJOs from the Schema, wrap the POJOs as DataObjects and use SDO  to link the DataObjects to the Schema.

In conclusion it seems that JAXB is more straightforward to use with an ecosystem comprised of a Java Web Services Stack (JAX-WS and JAX-RS) and a standard persistence mechanism (JPA). SDO has an edge when used with SCA (Service Component Architecture – a very powerful and modular component architecture) and when a rich meta-model is expected from the Java Objects. JAXB is the more popular of the two mechanisms and with the growing popularity of REST (and JAX-RS) it definitely has an edge. The only caveat is that not all implementations are created equal. You might want to check for yourself the most popular ones and test them, it’s as simple as changing one line in the jaxb.properties file:

• Glassfish Metro
• Apache JaxMeAPI
• Eclipse MOXy

JavaOne 2010 – Enabling Transformation Through the Cloud (a non-IT perspective)

This round-table gathered four KPMG consultants (Steve Hill, Egidio Zarello, John Cummings and Mark Foreman) to discuss the adoption of the Cloud from a business perspective. A few good points were made during the hour that the roundtable lasted, although these points could have easily been delivered in half an hour without any loss of information.

The main theme was about the Cloud being embraced more and more by the business units as opposed to the IT departments and that they don’t see this trend slowing any time soon. I have to say that I completely agree with their point of view; our own experience at Lab49 corroborate this  trend and not just in the Cloud adoption area. End-users are pushing for the adoption of a business enabling technology and IT can and should have a role in it. When asked to define that role John Cummings clearly said that this role was a governance one: Defining the privacy model, the deployment model, negotiating the SLAs, etc… What is meant exactly by “business enabling technology”? If the cloud is a technology allowing self-procurement and data-center virtualization then surely it can also virtualize business processes.

The discussion then veered towards the inhibitor aspects: The macro-economy as a whole may prevent certain places from experimenting with the cloud, security concerns, transparency and control over the data. But when one thinks hard about each one of those aspects it quickly becomes clear that they should not stand in the way of cloud adoption. The tough economical environment is actually a driver for the Cloud adoption to reduce the cost of data-centers, security concerns can be addressed with the proper governance model, transparency can also be addressed with the proper SLA in place, privacy is already a concern for corporations conducting trans-frontiers transactions. What’s left is data control. The roundtable did not address this issue and I think that most architects would favor a hybrid solution right now.

Finally the Cloud is being adopted all over Asia, with the most obvious examples being China and India; China mobile alone will spend 58 billion USD over three years (that’s not a typo) over their cloud infrastructure while India sees the Cloud as a way to “virtualize the nation”: Think about it, hundred of million of people have become urbanized over the past couple of decades (now that’s the real revolution of the end of the twentieth century, not the Internet, as Michel Serres has explained it at large) and Infrastructure as a Service (IaaS) is seen as the way to go to give them cheap access to technology. The US government is doing its share to push for Cloud adoption across various agencies: many agencies can now go and buy SaaS with a credit card instead of waiting for the two-year cycle of budget approval by Congress.

JavaOne 2010 – Patterns for modularity

This BOF featured Jaroslav Tulach, founder of NetBeans,along with Anton Epple and Zoran Sevarac. It was not really about new technology but about formalizing the approach and terminology for building modular systems. The talk targeted both desktop and server developers. The premise was that OO alone did not deliver on code-reuse hence the need to apply patterns, similar to the GoF patterns, to modules.

The speakers did make it clear that patterns exist within a context, i.e. some patterns might not be applicable to a given language, for example, since that language might already have constructs to provide the solution to the common problem addressed by the pattern; having said that the discussion centered exclusively on Java.

Anton defined his “Five Criteria for Modularity Patterns” even though last  I checked they were six (they were still fixing the slides as we entered the room…):

1. Maximize re-use
2. Minimize coupling
3. Deal with change (i.e. a smaller system deals with change easier than a larger one; JDevelop, for example, can make changes to its core easier than, say, Eclipse)
4. Ease of maintenance (each release should have a theme, such as release X of NetBeans will support OSGI; this well defined theme should not affect the existing system)
5. Ease of extensibility (how powerful and simple is your plug-ins architecture)
6. Save resources (your modules should not affect the start-up time, especially true for desktop systems, and the memory footprint should be kept manageable)

Another set of definitions followed to formalize the Relationships Dependencies Management:

1. A Relationships Dependencies Management is said to be direct if Module 2 depends directly on Module 1:            M2 -> M1
2. A Relationships Dependencies Management is said to be indirect if Module 3 depends on Module 2 which depends on Module 1: M3 -> M2 -> M1
3. A Relationships Dependencies Management is said to be cyclical if Module 2 depends directly on Module 1 and Module 1 depends directly on Module 2 (no need to draw a picture)
4. Relationships Dependencies can be defined as Incoming (M1 -> M2 <- M3) – they make M2 hard to change, or Outgoing (M1 <- M2 -> M3) – they make M2 easy to change
5. Finally Relationships Dependencies can be designed using three classical patterns: The Adapter pattern (Adapter Interface between 2 modules to introduce an Indirect dependency; it forwards method calls from M3 to M1), the Mediator pattern (sits in between 2 or more modules, aka the Bridge pattern in NetBeans) and the Facade pattern (provides a front Interface for a set of two or more modules)

The last portion was the more practical one as it provided an overview of the existing tools in the Java space to solve the problem of Reducing Communication Dependencies. The problem of Communication Dependencies can simply be stated as follows: Given an Interface TextFilter, an implementation class UpperCaseTextFilter and  a client Class Editor, how can Editor get an implementation of TextFilter? Ideally it should know nothing about  UpperCaseTextFilter at design time. The ideal run-time solution should provide the following (now we’re getting at the heart of system modularity):

1. Register a Service
2. Retrieve a Service
3. Disable a Service
4. Replace a Service
5. Order a Service (as in providing some sort of ranking)
6. Declarative support for a Service (as in meta data)
7. Codeless Service (as in configuration)
8. Availability of required Services

Five solutions were described; I would like to point out that SCA was left out which is a shame because it is a well thought technology and Apache Tuscany is only one of its many implementations.

1. JDK 1.6 own service loader mechanism; it is declarative (in META-INF/services) and returns an iterable typed collection of services but it is way too simple, it is not dynamic (you can’t react to a situation where the client uninstall a plug-in) , it can be dangerous as it loads all services at start-up and it does not provide factory methods
2. NetBeans solution: Uses Lookup and XML files. This one is declarative and dynamic, allows for ordering, for lazy loading, has factory methods and codeless extensions
3. OSGI Service Registry: Services are registered with code using bundleContext.register(), it is dynamic, it has factory methods, it filters services and it is configurable with code, which means that you now have dependencies on the OSGI framework in your code, eager creation can slowdown start-up times and it is not type safe
4. OSGI Declarative Services (OSGI II if you prefer): It’s better than Service Registry in the sense that it is declarative (XML configs)
5. Dependency Injection: Spring offers an alternative solution using @Autowired and it is declarative in nature (the wiring is specified in  your beans.xml) and the framework is usually transparent to your code

Jaroslav then discussed the hotly debated issue: Are Singletons evil? And the answer is “it really depends on the context” (recall the statement earlier in this post). In a Dependency Injection solution there are many contexts: The Application Context, the reckless contexts and the Session context. Singletons would then be viewed as bad given all these contexts and the false sense of security that they provide. But at the module level (say a jar) they can be helpful if they are carefully designed:

• The Singleton must be ready for use: No prior initialization code should be required prior to requesting a Singleton
• The Singleton must be injectable: We should be able to inject different Singleton implementations at run-time depending on the context (say DEV v/s PROD)
• Singletons are OK when used with the proper Service Loader/Lookup mechanism

Finally a few thoughts on performance were given: Since modular applications tend to be so large start-up time becomes critical so one should obviously avoid calling in OSGI bundle.start() for each jar because it is inefficient and because often time the jar is a 3rd party library that can’t be trusted. You are better off using a declarative registration method (as in an XML config file – interpret it and cache it); which is why JSR-198 falls short in the performance area. JSR 198 is indeed declarative (XML)  but you need to create a handler for each service, which slows the start-up time.

Again, this session did not break any new grounds but it helped organizing the ideas around modules, what to look for when evaluating different solutions and last but not least how to learn from a large system such as NetBeans that has been continuously evolving for the last 13 years.

JavaOne 2010 – OpenJDK BOF

The OpenJDK BOF was an informal Q&A session, attendees were free to ask JDK-related questions and Kelly O’Hair, Dalibor Topic and Mark Reinhold were there to answer. I think that this setup was appropriate for such a sensitive topic given the degree of anxiety of the Java community and probably the state of mind of the former Sun employees themselves. I will try to capture the most relevant  Questions and Answers.

1. Will the JRockit VM get open-sourced? No, the plans are to keep HotSpot open-sourced as Open JDK 7 and add to it some of JRockit unique features by mid 2011.
2. Will Oracle keep going through the JCP? The plans are to keep using the JCP for OpenJDK 7 features such as the Lambda project and even for Java SE 8. No other guarantees were made beyond that.
3. Comment on JDK 7 v/s Open JDK 7: Sun used to provide the OpenJDK + some of its own proprietary binaries (themselves largely based on the OpenJDK, such as plugins) gratis (as in free beer). Oracle will continue doing so in OpenJDK 7 and at the same time provide JDK 7 which should be 98% identical to OpenJDK 7.
4. Will Oracle make some performance-sensitive features (such as sockets, io) part of JDK 7 as opposed to OpenJDK 7? No because Oracle has little/nothing to gain from such a model. It would only defragment the  code base and make the merge back into an OpenJDK a nightmare. On the other hand JRockit Mission Control API will remain proprietary but some of its features will make it to the Open JDK such as the pluggable verbose logging and the JMX management tool (it can handle port numbers while the JMX on the HotSpot currently can’t).
5. Will the deterministic GC of JRockit make it to OpenJDK? No because there are currently paying customers that Oracle would like to keep as such. On the other hand certain aspects of the JRockit GC could make it such as incremental sliding compaction.
6. Will the Da Vinci code continue to thrive? Yes, unfortunately not all of John Rose great ideas can/will make it to OpenJDK 7. For now JSR 292 (Lambda project – closures) will make it. The rest, most notably support for tail recursion, will not.
7. What’s the state of Jigsaw? It’s in a state of flux, we’re not at a point where we can readily start portions of the JDK. But it is actively worked on for OpenJDK 7.
8. What about Project Verrazano? For now it is a research project (it takes a JAR and a platform spec, modularizes the classes themselves to reduce their code size to an optimal one).
9. What about OpenJDK 6? Actually the effort was started after the one for OpenJDK 7 and so it did not start with the Java SE 6 code base, rather it started with the OpenJDK 7 code base and engineers started removing features to match JDK 6. It slightly differs from JDK 6, depending on the particular repository but the main features such as CORBA, JAXB, JAXP and JAX-WS, when they differ, do so very little, and usually only in terms of exact licensing. Currently there are are only four different features between JKD 6 and OpenJDK 6: Graphics, Fonts (the most prominent one), SNMP and Color Management.
10. Speaking of repositories, which distributed SCM do you recommend? The Solaris team opted for Mercurial (Python-based), it’s great, we could have gone with Git but not sure what to do about the added complexity

So there you have it, the latest on OpenJDK 7; the session started slow with few questions but as time went by the audience asked more and more questions, mostly I think to get reassured about the open-source fate of OpenJDK. The message was indeed a reassuring one, but one thing remains to be seen is: What will happen to Java SE post 2012?

JavaOne 2010 – Enterprise Service Bus, Lessons from the field

Good presentation about the ESB adoption for a major web site, nfl.com. The two presenters, Earl Nolan and Monal Daxini, were eager to share their pain points during the adoption of Mule as the ESB (Enterprise Service Bus). Unfortunately Mule was the only ESB discussed, but during the Q&A the presenters admitted that Spring Integration would have been considered for a smaller-scale effort. They did not look at Apache ServiceMix because three years ago it was not quite as stable or feature-rich as it is today, so were they to evaluate the offerings today ServiceMix might have been adopted.
They started off by (aptly) saying what ESB is not: ESB is not JMS or a messaging middleware platform and ESB is not a heavy web services stack. They (also aptly) gave a simple definition of an ESB: A solution for integration problems. Put it simply, any time 3 or more applications need to integrate you have the potential for an ESB adoption. The work of Greg Hohpe was cited quite often during the course of the talk.

So why adopt an ESB to solve integration problems? I can think of a couple of other ways myself where an ESB is not warranted/desirable. There are quite a few cases where an SOA solution or an OSGI-based solution are more desirable but it is clear that ESB is relevant to solve problems where a large number of applications need to talk to each others and along the way transform/enrich the data in some way. An ESB allows you to decouple the integration logic from the business logic (more on that later, but think for now configuration over coding) and evolve from a hub-and-spoke or point-to-point communication model towards a more distributed model.  Finally an ESB has typically an impressive list of connectors that can alone justify its adoption (we are not talking JCA here…)

So why choose one ESB vendor over another? Here are a few tips from this session:

• Go for Configuration over Coding – actually that’s the whole idea of an ESB, eschew fancy APIs in favor of fancy configurations. Later we’ll talk about when you might want to use the API
• Go for a lightweight ESB (quick start-up time and easier to test)
• Go for an ESB that favors (and properly document) an incremental adoption of the product – that’s just common sense but often time the documentation offers few clues on how to partially get started with ESB
• Scalable and distributed (had to mention this one…. one more bullet… but hey, it’s really important)
• Embeddable – what’s meant here is that the ESB should be able to reside (maybe in a first phase) inside your existing process to ease deployment and simplify dealing with the IT ops team
• And finally go for some type of SEDA architecture – we’ll go over this point in details

What’s SEDA? It stands for Staged-Driven Event Architecture (come to think about it, it’s quite reminiscent of the EDA model I encounter in CEP applications) and it’s a programming paradigm where the application is decomposed into different stages, each stage is fronted by a Queue and each stage (except the last) feeds onto the next one. An analogy can be made with microprocessor stages and their pipelines: Stage 1 would pipeline new inputs into the queue if stage 2 is not done processing yet. This approach avoid complex concurrency problems. Mule has wholeheartedly adopted the SEDA model by providing a Queue Controller at the input of the system, the system is made up of stages, each stage contains an Event Handler and the developer is only responsible for implementing the event handlers. The thread pool, the Queue Controllers (including the one that provides feedback to slow down/stop incoming messages) are implemented by Mule which instantiates as many Eventy Handlers as required. The queues themselves enable modeling and capacity planning.

Next the speakers described the stage itself since it is at the heart of the SEDA model: Each stage is fronted by an input (queued) channel and outputs data through a (queued) outbound channel; inside the stage data goes successively through an inbound router, a service component and an outbound routers; transformations are handled by the routers. What’s interesting to note is the number of protocols that an ESB such as Mule can handle at the channels: JMS, FTP, File system, HTTP, UDP, TCP, IPoAC, etc…

Next was the paradigm shift introduced by ESB; this is actually similar to any mentality shift that a developer must undergo when adopting a new technology, and again Greg Hope was quoted: Your coffee shop doesn’t use 2-phase commit! Mainly don’t fall into the Leaky Abstraction trap and do make use of the ESB components such as splitters (the net effect for the developer is the ability to deal with a single thread).

Next was a series of don’t do:

• ESB is not a pass-through proxy: what they meant is that given all the layers involved in a typical ESB it does not make sense to use the ESB as a simple pass-through if there is no-value added (i.e. transformation, projection, etc…) It simply complicates the stack and you will have to think of a myriad of problems such as caching, host down-time, etc… You are better off using a CDN for that.
• ESB is not a glorified CRON job scheduler – use a cron tool for that (this applies to shops with a heavy reliance on batch jobs)
• EASB is not an application glue; use a Dependency Injection framework instead

Finally a few best-practices were presented such as the separation of validation from transformation (use an ETL plug-in if needed for complex transformations – I am not sure about this advice, ETL left me with a bad taste even when used topically) and enforce data canonicalization – I think that this last point applies to most platforms, whether developing web services, ESB or OSGI. It’s worth repeating it: Time spent defining a canonical model for your data is time well spent and will save you from redundant validation/transformation/exception flows down the line.
About data validation: The actual way to go about it is quite controversial, do you go with a strict (as in XSD schema) model or with a more relaxed model (RelaxNG and Schematron)? I don’t think that there is a clear-cut answer, it really depends on how dynamic your environment is and how many external dependencies you have (for Internet-facing applications the relax model is probably better).

About your event model: Again two models were discussed, push v/s pull but here it’s easy to see why a push model would be preferred unless you have very stringent reliability requirements I can hear the Nirvana folks scream since they do have buffering/replay capability). The low latency/low cycles consumption features of the push model makes it a clear winner but it is not always enforceable when dealing with 3rd party data providers.

The session wrapped up with security and deployment issues.

It’s interesting to note that most recommendations are applicable to most development situations, and that’s what reassuring about the ESB adoption: There is nothing fundamentally awkward about the model, just a formalized way of doing data integration that forces you to modularize your aspects, make the proper architecture choice (validation, push/pull, transactions or not) and decouple the integration logic from the business logic. All in all an entertaining talk, one of many that dealt with ESB, showing that this technology is very much relevant whether you operate in an EDA environment, an SOA stack or a more traditional back-end system.