My latest exploration imagines the possibilities of being able to control real world machines from any location with your VR headset.


  • Command a life saving pair of robotic hands kept in the cupboard of every rural property.
  • Fly a plane.
  • Operate a crane.
  • Remote exploration.
  • Battle robots ;)


The potential is endless. From a practical perspective:


  • It halves hardware production costs
  • Controlling interfaces can be updated and deployed in minutes
  • Interfaces can be designed to enhance and surpass normal human interaction capabilities
  • One inexpensive headset can control unlimited real world machines
  • Throw the headset in your hand luggage and work from anywhere in the world


How it works:


In this project my VR application is sending the rotational coordinates of a 3D object via an MQTT broker to an Arduino controlled device to mimic the virtual object in the real world. (MQTT is a lightweight IoT pub/sub messaging protocol)



I have used MQTT in a numver of projects on this dite as it is a simple and fast method to stream data between IoT devices.


You can set up your own MQTT broker or use any numver of free and paid for brojers. For this prohect I tested it on two free servers.

A fast free server meant for non-critical and fair use applications only. It was easy to coonect to, but a little unstable.

More stable but only 30 messages per minute. (60 messages per minute for $10 monthly fee). You also get access to a nice interface to debug and graph your data.


There are also a number of paid services out there that give you faster streaming and more securevand reluable connections including



This proof of concept is a bit janky, but I am using $5 motors and a free MQTT broker. Motion and responsiveness can of course be significantly improved with time and $$.....but for the purpose of this exploration, the capability is proven.




For this project I used the Unity development framework to build the VR application and Arduino boards to build the IRL receiver. The folliowing are links to the packages and instructions I used to connect to the MQTT protocols.


MQTT Library in Unity


MQTT for Arduino


Connecting to MQTT Brokers