Messaging and Infrastructure Management

by andy

I’m reposting this article I authored (edited by John) for the SLA@SOI website.

SLA@SOI is dedicated towards building an SLA-aware service oriented infrastructure. A key component of this will be the layer managing the infrastructure, and we are envisioning that this must be able to handle internet-scale deployments. Management of an environment with hundreds of thousands of nodes places significant demands on communication channels, and SLA@SOI is currently investigating how protocols like XMPP[1] might help address these needs.

There are many discussions debating the pros and cons of XMPP [2][3][4] as a messaging protocol in this type of environment, some arguing how AMQP [5] might be a better choice [6][7][8], and others comparing HTTP versus XMPP [9][10]. Indeed, Roy Fieldings makes the point in one of his blog posts [11] that HTTP can be equally suitable to XMPP so long as an application’s constraints and requirements are fully considered. This might hint that higher-level consideration be given when designing the architecture of a system and perhaps not to be caught up in the all important detail.

In SLA@SOI we are targeting the management of hundreds of thousands of systems in a heterogeneous and dynamic environment. The infrastructure may include both physical and virtual nodes. The size of such a system means that interaction via traditional RPC (REST, SOAP calls) or stateful or persistent connections would soon run into scalability problems. Heterogeneous aspects and requirements of such systems calls for open and simple protocols that support a high degree of interoperability. To support highly dynamic infrastructures, publish/subscribe mechanisms offer substantial advantages [12], assuming of course that all systems can talk the same messaging semantics and protocol.

For efficiency, SLA@SOI also perceives the need to communicate with multiple nodes at the same time, in a multicast fashion. Although multicast semantics can be mapped to HTTP, this is more of a workaround than a feature. For our needs, based on our constraints, we have concluded an event based architecture to be the most appropriate, and the semantics contained within the PubSub meta-pattern (referred to as so due to the large number of sub-patterns [13] within) to be very attractive and suitable.

Rather than lock ourselves into a particular implementation of PubSub (XMPP, AMQP, JMS etc), we have decided to abstract the PubSub layer through the use of interfaces and implement a pluggable PubSub framework which would support the integration of any PubSub implementation. By settling on a “standard” interface to PubSub the remaining concern to us is of the data schema that represents messages passing through the chosen PubSub implementation.

There is the danger of designing “a schema to rule all schemas” – one which once agreed upon would then become instantly brittle and inflexible. We are considering to take the learnings of REST (with its limited finite set of verbs operating on resources) and layer that on top of an extensible PubSub event based system. This event based system would allow the use of arbitrary PubSub protocols and corresponding implementations. Regarding the selection of protocol for SLA@SOI, we are investigating the XEP-060 PubSub [17] extension to XMPP, and so far our results are largely encouraging:

  • XMPP’s open protocol and XEP mechanism ensure flexibility.
  • PubSub is powerful and allows the transport of opaque payloads (there are pros and cons to this, however it is very flexible).
  • XMPP has widespread support from other messaging solutions – we can bridge from XMPP to JMS or indeed many ESBs.
  • Multi User Chatroom (MUC) and direct one-to-one messaging are ideal for a means of debugging, offering “command line” interaction with XMPP nodes.
  • The instant messaging (IM) model is very understandable – people can relate to it. They just need to think beyond the notion that XMPP is a chat protocol, and see it as general purpose messaging protocol that is easily and openly extensible.
  • XMPP seems to be nicely scalable (just choosing XMPP doesn’t imply instant scalability – an appropriate architecture is critical too).
  • S2S is very attractive from the point of view of enabling resource federation. This should be of particular interest to Reservoir [16], a fellow FP7 project.

We have also identified some challenges with XMPP…

  • We have not found any “all-in-one” distributed (as in P2P or super-peer type) implementation of XMPP, although the specification allows for such interactions (S2S) – possibly Vertebra[15]
  • XMPP PubSub XEP implementations have inconsistencies (core features and XEP coverage)[7][14] and client libraries appear to be thin on the ground (particularly in Java)
  • Routing, workflow management seems to be lacking in XMPP.

In SLA@SOI we will continue our investigations into messaging at the Infrastructure Management layer, focusing on extending PubSub with REST semantics via possibly the Adhoc-Command [17] and IO-Data XEPs [18], and will be happy to update you on our progress as the project progresses. If you have any thoughts or opinions on the matter please do let us know!

References: