It is already possible to get robots to swarm and communicate, but will it be possible to take the next steps and make bots that can metamorphose with or without intervention?
A boy shows off a tiny, penny-sized object. He flicks it to the floor, where it's swiftly joined by hundreds of identical objects to form a large box. It disintegrates and morphs into a giant hand that gives a wave; then into a bridge, a building.
Right now this is still a scene from a computer animation, the children's movie 'Big Hero 6', featuring microbots that shape-shift to form 3D structures on command. This is 'programmable matter': halfway between self-moulding clay and self-assembling machinery, and definitely a leap beyond 3D printing, with bots that can be programmed to change their physical properties.
Obviously we are not quite there yet, and smart liquid robotics – in the style of the'evil T-1000 robot in the movie 'Terminator 2: Judgment Day' – is even further out. Still, programmable matter is creeping closer.
"Today most robots are – for a lack of a better word – dead," says Hod Lipson, a robotics engineer and the director of the Creative Machines Lab at Cornell University in Ithaca, New York. Bots of the future will be self-replicating and self-repairing, "closer to biological systems and able to adapt, improve, recycle, and evolve".
It is already possible to get robots to swarm and communicate – whether it's military drones that could 'swamp' an enemy aircraft, or smart cars that warn each other to avoid traffic jams. Researchers have also managed to get tiny bots to self-assemble into 2D shapes. Scaling them down and going from 2D to 3D is the next step – part of a programmable matter concept known as claytronics.
Another approach to programmable matter is introducing programming into materials themselves, making them respond to external stimuli such as heat or water in order to change shape.
If it works, it will have a huge impact, creating an artificial reality to interact with. You won't need headsets or smart gloves, says Seth Goldstein, a computer scientist at Carnegie Mellon University in Pennsylvania, who together with his colleagues coined the term 'claytronics' in 2002.
Clad in white overalls, he examines a petri dish in one of many clean rooms at Carnegie Mellon. But it's not frail fruit flies he's here to dissect. Instead, the dish swarms with tiny, metal-like objects.
These are 'catoms', short for claytronic atoms: tiny robotic cylinders made of micro-electromechanical systems or MEMS, powerful machines just microns in size, slightly larger than the width of a human hair. They use electrostatics to communicate, cling together, and transfer energy to one another. "Getting them to compute, and as a result self-assemble, changing the overall structure's physical properties, is the next step," says Goldstein.
If he and his colleagues succeed, objects could soon be put together (or repaired) on the fly; no more Ikea wardrobe assembly stress. A carpenter could scoop up a lump of claytronic goo from his kit, shape it into a hammer, a wrench, or a spare part that will morph to fit and repair anything he needs to fix. Clothing would change its thermal insulating properties in tune with the weather. A soft robot might flow like mercury through tiny gaps to enter hidden caves and underground bunkers. An aircraft could change its wings mid-air to adapt to the flying conditions. You would interact with a morphable, 3D interactive life-size TV in your living room.
It's 'the ultimate adaptable material', as Mitchell Zakin, a scientist who worked on programmable matter for the United States' Defense Advanced Research Projects Agency (DARPA), once called it.
Even though the technology sounds really futuristic, Goldstein is convinced that we could have functioning prototypes within a decade – provided research perseveres. "There are no challenges for which we don't have at least hints of a solution. It's all about what makes you tick – and for me, it's this," he says.
From ants to bots
Goldstein isn't the one who pioneered the programmable matter concept though – nature did. A peculiar single-celled organism called slime mould under stress spontaneously morphs into a multicellular body without any central brain telling it to do so. An individual fire ant may not be particularly bright, but a colony of them assemble into live sturdy bridges across gaps in their paths and make hydrophobic rafts to survive floods by locking their flimsy-looking arms together – even without a leader. Or even take the human brain – each single neuron can only react to what the neurons next to it are doing, but all of them together can be Albert Einstein.
The term 'programmable matter' was first used in 1991 by MIT computer scientists Tommaso Toffoli and Norman Margolus. They referred to a cluster of tiny computing elements, cooperating by using nearest-neighbour interactions. The idea was to create a computer that could mimic the physics of real matter. But other researchers, including Goldstein, thought of programmable matter as an assembly of tiny machines that could locally change positions to alter the shape of the overall structure.
In collaboration with Intel Research Pittsburgh, Goldstein's team in the early 2000s was among the pioneers developing programmable matter prototypes. The first attempts were macroscopic – cylinders a bit bigger than 44mm in diameter. Inside were electromagnets that made the cylinders stick to one another. When the researchers switched the magnets on and off, a catom would wriggle its way around its neighbour.
Now Goldstein is busy scaling the system down, hence the petri dish. The main challenge, he says, is in programming the software for the massively distributed system that will result from assembling all the bots. "We need each catom to know the positions of its immediate neighbours. Once they are all programmed, the bots will be able to find the right configuration on their own, by 'communicating' with one another using electrostatic mechanism for nearest neighbour sensing," says Goldstein.
For catoms to assemble in such a way, Goldstein's idea is to fill the initial claytronic goo with lots of tiny cavities and then move the catoms around, shifting the voids to get the right overall shape. Once the gaps reach the surface, the structure would shrink, and if the material opens up pockets at the surface and consumes them, it could expand.
Carnegie Mellon is one of several universities pursuing programmable matter; Goldstein has also been working closely with a team from the Femto-ST Institute in Besançon, France, led by computer scientist Julien Bourgeois. The French researchers are already looking beyond the lab; recently they forged a deal with a carmaker Peugeot Citroen to use claytronics for prototyping and design.
The idea is to have a computer-aided design (CAD) tool – computer software that supports the design process – connected with a bunch of catoms. Once a part is designed, the information is transferred to the catoms, which then take the shape of the part, turning into a physical prototype made of tiny bots. "The designer can then directly interact with the matter, touching the catoms he wants to move, like electronic clay," says Bourgeois. Once the designer is happy with the result of his smart dough-playing, the data gets uploaded on to the CAD again to produce the final part to the precise parameters determined by the manipulations of the blob of catoms.
If it works, the claytronics tool will help carmakers save time and suggest more prototypes during the conception stage, says Peugeot Citroen's deputy scientific director Stephane Delalande.
For that to happen though, the bots will have to be able to reach by themselves a specific target, without a centralised 'brain' telling them how to do it – a much harder task than during simulation, where the computer is the boss. "The difference lies in the degree of embedded 'intelligence'," says Bourgeois.
Researchers at Harvard University led by Michael Rubenstein have managed to do just that with a swarm of more than 1,000 identical 3cm-sized 'Kilobots' that self-assemble into 2D shapes. Each bot is programmed with an image of the required shape, and then they work together to create it – just like cells forming organs.
To do so, each Kilobot – dubbed so because there are 1,024 of them, the same as the number of bytes in a kilobyte – has to make its own decisions based on what its immediate neighbours are doing and on its individual current state. Individually they are not particularly clever, but together, slowly, over six to 12 hours of shuffling their way on three skinny legs and flashing an infrared light to communicate with their peers, they amass into a pre-programmed shape. In future, such swarms can take 3D printing to a new level, says Rubenstein – and each tiny bot can be made cleverer too, with sensing, computation and actuation capabilities.
Another team at the Massachusetts Institute of Technology has been focusing on developing 'self-sculpting sand' – minuscule, grain-sized machines that join to form a 3D object and then fall apart again. "Suppose, for example that the robot needs a screwdriver from a shelf; the screwdriver is too high to reach. What if the robot could grow an extra-long arm to get that screwdriver?" says one of the researchers, computer scientist Daniela Rus.
She and her colleagues have created some 30 prototype 'smart pebbles', each about the size of a sugar cube. Run by specially designed software, they communicate with their neighbours to self-assemble into 2D shapes and stick together thanks to electropermanent magnets embedded into their sides.
But programmable matter isn't limited to creating a swarm of bots. Another approach is a step beyond 3D printing and going for 4D printing instead – creating smart materials that can be shaped and re-shaped. This, again, is inspired by nature: at times spontaneously, at times in response to a stimulus, cells may change their properties.
"Usually, materials are 'dumb'," says Lipson. "Now we are moving from printing 'dumb' material to printing smart material that can react to its environment in intelligent ways."
To achieve this, information must be integrated into the material itself – with the help of embedded electronics, batteries, sensors and actuators. Imagine sticking a sponge between two sticks. If the sponge is dry, everything is straight, but if you increase humidity, the ensemble will bend.
Together with a team from Harvard, Rus has developed a 'self-folding origami' machine – a robot made of flat sheets of material with built-in actuators and data. In response to heat from an embedded electrical circuit, tiny folding hinges on the sheets contract, making the robot fold itself into a 3D shape. In future, such 'paper robots' may be able to turn into anything you want, says Rus: "a cup, a hammer, or a hat, taking the Swiss Army knife to the next level".
Apart from heat, researchers are trying to find other ways of 'programming' 3D-printed materials so that they become more robot-like, with the aim of getting rid of the need for electro-mechanical devices that require energy and could fail. For example, MIT computer scientist Skylar Tibbits is looking into creating materials that change shape – like sponges – when submerged into water, and 3D printed sensors that alter their colour when exposed to specific light sources thanks to embedded nanomaterials. This could lead to pipes controlling flow or pressure and building structures that self-evolve over time and adapt to changing environmental conditions.
Although this could be cheaper than claytronics, Bourgeois says that 4D printing will produce deformable but not 'programmable' material – at least not in the same sense as catoms would be. "A claytronics ensemble is able to run a code – but 4D printing won't," he says.
In all likelihood, the term 'programmable matter' will encompass a broad range of solutions, from microscopic catoms to self-folding origami machines, and even new materials like 'super composites' that can be programmed to be rigid in one direction and flexible in another, says Gord Kurtenbach, head of the research group at design software firm Autodesk.
Of course, he argues, we already have access to fully programmable materials and could take the building blocks for it from nature, thanks to synthetic biology – for instance, microstructures can be made just like a virus from DNA. After all, DNA can be programmed to assemble into specific structures, so all we need now, says Kurtenbach, is to "build tools that allow us to design specific functions".
This approach would take us beyond the limitations of 3D and even 4D printing, from personalised drug design to creating materials that grow or adapt as needed. "3D printing has a finality to it, whereas organic things grow and adapt over time, so that's a really different paradigm," says Kurtenbach. What if we could 'grow' a geothermal piping system, directly from a 'seed' and then program it to adjust as required?
Whether electronic or biological, the technology of claytronics and programmable matter is rapidly maturing. Hod Lipson at Cornell University is certain that it will deliver on its promise: "In the very long term – centuries from now – I think that modular systems will be the only way to ensure sustainable production."