HyperCat: Interoperability on the Internet of Things

27 June 2014
By Edd Gent
Share |
A truly global Internet of Things needs interoperability, but creating a framework to achieve this is no easy task [Credit: @wilg]

A truly global Internet of Things needs interoperability, but creating a framework to achieve this is no easy task [Credit: @wilg]

Annoying buzzword or not, the rise of The Internet of Things (IoT) is undeniable. But unlike the heady days of the early Web big corporations have got a head start and they are building it around their own devices, throwing up a mishmash of vertically-integrated systems that refuse to cooperate. Is there a way to break down these barriers?

The challenge is interoperability, allowing devices to understand each other’s’ data regardless of location, manufacturer or format. Much of the work so far has been focused on semantics – coming up with a common set of categories into which all IoT data can be shoehorned to make it easier for devices to know what they are looking at when they encounter new resources.

This kind of top down approach is fraught with problems, such as how to decide on ontologies – shared vocabularies – what language they are in, how you convince hundreds of operators to reorganise their data and not least how you convince everyone that out of all the possible frameworks your solution is the best. Needless to say, progress has been slow and in the meantime big firms like Google and Apple are pressing ahead with their own proprietary visions of the IoT.

But what if a less heavy handed solution exists? A consortium of more than 40 UK companies has been taking part in a 12 month project funded by the Technology Strategy Board (TSB) to the tune of £6.4m to solve the problem of interoperability, and yesterday they announced their solution – a new IoT specification called HyperCat.


The ‘soft standard’, as some of the projects backers like to refer to it, allows any hub of IoT data to describe the resources it holds in a common way, giving each resource a set of user-defined semantic annotations. Data hubs can then be interrogated to discover what data they can offer and they will return specific pointers to that data in the form of URIs (uniform resource identifiers). But importantly, the specification allows users to choose or invent any set of semantic annotations they want rather than imposing specific categories and labels.

When the consortium first sat down to look at the problem of creating interoperability, the response was predictable. "The first thing everyone said was, ‘Well, interoperability is easy, just look at our platforms’," says Pilgrim Beart of CEO of IoT specialist 1248, who was involved in the consortium.

Looking at everyone’s platforms did reveal a lot of similarities though. Nearly all of them used the same standards for organising their data – JSON for data formatting, HTTPS for security and RESTful for architecture. As a result the HyperCat specification supports these same open standards, but creates a common way of organising them so that data from various sources is visible to any app capable of understanding that kind of data.

Currently, the variety of different systems for organising IoT data means that a great deal of human intervention is needed. Apps are unable browse for data they can use, because they are only able to understand data in the formats they have been programmed to be familiar with. For an app to make use of a new data source a developer is required to write code enabling it to discover the resource – in much the same way mobile app developers have to write different versions of their app for iOS, Android and Windows Mobile.

With the number of connected devices having overtaken the number of humans several years ago, the level of human intervention needed to create a truly interoperable IoT is unfeasible under the current circumstances. As Beart notes, “Humans don’t do scale”.

“The opportunity and the challenge is that we as individuals can’t give these machines as much attention as we used to because we don’t have time. So the machines are going to have to look after themselves to a scale they’ve never had to before,” he adds.

HyperCat offers a potential solution to this problem. By making IoT data machine-browseable through the creation of a simple and malleable way of organising data it takes humans out of the equation and allows applications to find and exploit new data resources autonomously. So what’s the catch?

Thin on the ground

The answer is there are several, but first and foremost is the lack of structured semantics. The HyperCat specification creates a way of describing data resources, but it leaves the actually ontologies used up to users. So if a user uploads a data resource from a temperature sensor and states in the metadata that the data format is '°C', the app will need to understand the format '°C' to be able to find the resource. As the semantics are set by individual users it is possible that two developers will describe similar data differently and their resources will be invisible to each other's apps.

But what the specification does do is allow an app to find any data it does understand in any HyperCat enabled service, as they use a similar organisational structure. For example an app that plots the position of street lamps using their geo-location data, could just as easily plot the position of a motion sensor in someone’s home that has its geo-location data catalogued. It wouldn’t be able to access any of the functionality of the sensor, but it would be able to make use of the information it understands.

This lack of a defined semantic structure is actually the project's greatest strength, according to its backers. “The advantage of keeping it very lightweight is that that you have a very low barrier to participation,” says John Davies, head of semantic technology at BT, who was also involved in the project. “The deeper you go, the more intelligence you offer but the harder it is to get widespread adoption.”

Enforcing a strict semantic structure on developers is liable to put them off and is also antithetical to the nature of the IoT, where the categories of what can be added to the network are limited only by your imagination – the Internet of Pigs being a case in point. "HyperCat has not tried to do what other standards have tried to do which is specify everything," says 1248’s Beart. "It's a completely impossible task."

For the technology to become useful, a more defined semantic layer will be required so that the application plotting street lamps doesn’t keep plotting people’s motion sensors instead. Indeed, as Beart points out, those already using the system have started to adopt common ontologies because they want the interoperability this offers. The next stage for the HyperCat project will be to come up with best practice guides for semantics and data formats, but importantly this will be an optional layer and users wishing to use their own idiosyncratic semantics will be able to.

A crowded market

Removing barriers to participation is a wise decision on the part of the project's backers. As is to be expected of a cutting-edge area of computer science, the IoT has spawned countless standards, specifications and start-ups all with conflicting ideas about how things should be done. “A big problem is going to be adoption,” says Dr William Webb, deputy president of the IET and one of the founding directors of machine-to-machine specialists Neul.

“There are no end of standards related to the IoT. They don’t all do what HyperCat does, but still they are trying to gain acceptance in a world where there is deep confusion over all the different standards.”

And while for the IoT evangelist interoperability is an obvious goal, for some of the bigger players in this crowded market it’s nothing more than a threat to their market share. Companies like Apple, Google and Vodafone have got big chunks of the IoT market firmly in their sights and may be reluctant to give away the opportunity to lock consumers into solutions that only cater for their products and those of their partners. A world where your choice of smartphone determines your choice of home security system, washing machine, tennis racket and even car is not so far-fetched.

An event to announce the successful completion of the first step of the project was held at the House of Lords in Westminster yesterday. Hosting the event was the Earl of Erroll, Merlin Hay, who has a background in technology and a particular interest in the promotion of interoperability. Talking up HyperCat at the event he said, “It enables a smart consumer that can say I don’t want a solution that locks me into that silo”. But to sway the consumer, the specification needs to be able to offer an alternative to that silo, which will require widespread adoption and a global reach.

Private and proprietary data

Another problem is the extent to which data will be accessible to applications. The HyperCat specification anticipates that users will want to restrict access to some data and adopters can use basic access authentication over HTTPS to control who can see what. But if huge amounts of data are likely to be restricted then the usefulness of a fully connected and interoperable network begins to diminish.

The IoT devices making the news tend to be geared toward the private customer. Smart meters, home automation devices, health monitors, smart watches and connected cars all seem likely to be creating highly personal data that users are unlikely to want to share. The rise of machine-to-machine communication in industrial control equipment is another major growth area for the IoT but you can see why manufacturers might want to keep this data to themselves.

“I don't suppose it's likely an energy supplier is going to let anyone query anyone else's electricity usage data,” says the IET’s Webb. “I think a lot of datasets intrinsically have privacy or security issues that might make this kind of open discovery somewhat irrelevant.”

Despite these problems several non-TSB-funded projects have adopted the specification, including the MK:Smart smart city program in Milton Keynes. The more than 40-strong consortium behind the project is further divided into another eight consortia each with their own idiosyncratic approach to using and testing the specification. Now that the first phase of the project is over, the TSB plans to pick one of these consortia to take the project on with a further £1.6m in funding.

For those hosting data hubs using the specification, potential business models range from companies providing free access to data for the public’s benefit, firms licencing their data to customers or even brokers who aggregate data from multiple sources to sell on. Part of the next stage of the project will look at creating standard licenses for data resources that could allow machines to automatically tell what data they are allowed to exploit, further removing the need for human intervention.

And while there are undoubtedly obstacles to adoption, the simplicity of the system could prove its saving grace. BT’s Davies oversaw the conversion of BT’s proprietary system to a HyperCat enabled one and he is adamant that the development work involved was minimal. And while the big players may fight their corners for as long as they can, the inexorable push for better and more holistic services may eventually make the case for interoperability overwhelming. “There will never be only one hub, that’s why HyperCat is crucially important,” says BT's Davies.

Share |
Related forum discussions
forum comment To start a discussion topic about this article, please log in or register.    

Latest Issue

E&T cover image 1607

"As the dust settles after the referendum result, we consider what happens next. We also look forward to an international summer of sport."

E&T jobs

More jobs ▶


Choose the way you would like to access the latest news and developments in your field.

Subscribe to E&T