The Big “O” is coming

I’ve blogged elsewhere about my dismay at fads in IT: “Enterprise Architecture”, and “Big Data” being a couple that I may have mentioned. Today I am recording what I think is going to be the next big fad, but one that I believe is becoming increasingly necessary.

I’ll name it now rather than keep you in suspense for any longer … “Orchestration”

So what is driving this?

The need to “Digitalise” our businesses, whatever that means, for one. Whether we understand it or not, “Digital Business” is something that we cannot afford to ignore. For once, we in IT are not making this problem but instead the circumstances have come together to make this emergent. If we don’t react, then others will and we’ll be left behind. Digitalisation calls for us, amongst other things, to drive new levels of automation, and to be far more reactive than we have been in the past.

But at the same time we are making our IT and computational needs increasingly complex. Dr James Cuff describes some examples in his article at The Next Platform called “Striking Practical Computational Balance“. But you don’t need to look at the big problems like genome sequencing, a lot of our routine IT tasks are also becoming more complex.

And so we face a classic dilemma: do more, but with the same resources or less.

What becomes important is our ability to understand what resources we need, and what we have available, and then make intelligent decisions about what to actually do. We may be able to fulfil all demand put on us, or we may have to make some hard decisions. And in trying to fulfil that demand, we will need to know when we are running out fo something and be able to call for more

And this makes for some challenges for IT

By day I ply my trade in the IT team of a “Telco”, or a Communications Services Provider. In the telco space, we are starting to become familiar with “orchestration” as a way to deliver contemporary network services. The multiple levels of operation make these much more complex to configure. New orchestration products such as OpenDaylight are seen as a way of using the differing levels of network resources in combination to build the flexibility needed to configure services.

Similarly in DevOps we see products like Kubernetes which, despite having been 15 years in gestation, is orchestrating the running of software containers in a way to make the most use of precious compute and storage resources.

The good news is that the wider problem of wanting to do more with minimal or no increase to resource needs is starting to be recognised. This is initially coming from the large cloud providers who build IT infrastructure on a massive scale, and can see the savings and benefits of driving efficiency. As the stories from Microsoft, Google, Amazon et al are shared, then the desire to make relatively similar efficiencies in our smaller IT organisations grows.

But the technologies that can be used here are nascent, and apply to very specialist fields. We do have elements of the fad-ism mentioned at the top of this post as the word “orchestration” appears to be taken at face value by a lot of people in the IT and Communications industries, who then misapply it and cause more confusion: “You want to get efficiencies by orchestrating? Use my product X. Guaranteed…”.

The truth of this is that to get the real business benefits today we would need to use a mix of different orchestrator products together if we are to have end-to-end orchestration generating real benefits, and that leads us back towards the compute complexity area as described by James Cuff. I am aware of an organisation that has implemented levels of orchestration, at the top level having an “Orchestrator of orchestrators”, but progress deploying this has been slow. Furthermore it still only addresses the technology, and the cost of entry of deploying a mix of these specialist tools can be prohibitive.

My prediction here is that we will start to see business drivers becoming more prevalent. That in turn will hopefully lead to us looking at “Business Orchestration” as a multidisciplinary subject, to be followed by frameworks and products that can be more widely applied and integrated into business operations. This will require new business metrics, and ways at looking at how our organisations operate. The good news is that there is something already in place that could easily form the basis for this: Stafford Beer’s Viable System Model

Watch this space