Introducing Vertex

Vertex is (will be) an implementation of Web Thing API specification by Mozilla IoT. In layman’s terms it will be the software package which will run and power the Internet of Things (IoT) edge nodes or devices. The name ‘vertex’ signifies an edge running on the internet. Many such ‘vertex’ nodes will run together in a network to form an Edge Computing Network and provide services in a de-centralized fashion on the internet.

The rest of the post will cover how to build, install and run vertex on a node. Please note that vertex is still under active development and the prototype will miss many specifications. We will keep on updating this post as and when we add new features and capabilities.

Dependencies

  • python

Build Instructions

  • Get the python dependencies
$ pip install -r requirements.txt

Running the Thing API Server

$ python vertex.py server [port]

Importing Web Thing Devices

  • Add a schema file
{
  "@context": "http://iot.schema.org",
  "@type": "Toaster",
  "name":"Acme Toaster",
  "description": "A web connected toaster",
  "properties": {
    "on": {
      "type": "boolean",
      "description": "Whether the toaster is currently heating bread",
      "href": "/properties/on"
    },
    "timeRemaining": {
      "type": "number",
      "unit": "seconds",
      "href": "/properties/timeRemaining"
    }
  },
  "events": {
    "ready": {
      "description": "Your toast is ready!"
    }
  },
  "actions": {
    "pop": {
      "description": "Pop up the toast",
      "input": {
         "type": "number"
       },
      "callable": "echo Popping up these many toasts"
    }
  }
}
  • Import using the vertex executable
$ python vertex.py import toaster /path/to/toaster-schema.json

Using the Web Thing API

  • Get a list of all things
$ curl localhost:3000/things
 
[
  {
    "actions": {
      "pop": {
        "description": "Pop up the toast",
        "href": "/things/toaster/actions/pop",
        "input": {
          "type": "number"
        }
      }
    },
    "description": "A web connected toaster",
    "events": {
      "ready": {
        "description": "Your toast is ready!",
        "href": "/things/toaster/events/ready",
        "type": null,
        "unit": null
      }
    },
    "links": [
      {
        "href": "/things/toaster/properties",
        "mediaType": "application/json",
        "rel": "properties"
      },
      {
        "href": "/things/toaster/events",
        "mediaType": "application/json",
        "rel": "events"
      },
      {
        "href": "/things/toaster/actions",
        "mediaType": "application/json",
        "rel": "actions"
      }
    ],
    "name": "Acme Toaster",
    "properties": {
      "on": {
        "description": "Whether the toaster is currently heating bread",
        "href": "/things/toaster/properties/on",
        "maximum": null,
        "minimum": null,
        "type": "boolean",
        "unit": null
      },
      "timeRemaining": {
        "description": null,
        "href": "/things/toaster/properties/timeRemaining",
        "maximum": null,
        "minimum": null,
        "type": "number",
        "unit": "seconds"
      }
    },
    "type": "Toaster"
  }
]
  • Get details for a particular thing
$ curl localhost:3000/things/toaster

{
  "actions": {
    "pop": {
      "description": "Pop up the toast",
      "href": "/things/toaster/actions/pop",
      "input": {
        "type": "number"
      }
    }
  },
  "description": "A web connected toaster",
  "events": {
    "ready": {
      "description": "Your toast is ready!",
      "href": "/things/toaster/events/ready",
      "type": null,
      "unit": null
    }
  },
  "links": [
    {
      "href": "/things/toaster/properties",
      "mediaType": "application/json",
      "rel": "properties"
    },
    {
      "href": "/things/toaster/events",
      "mediaType": "application/json",
      "rel": "events"
    },
    {
      "href": "/things/toaster/actions",
      "mediaType": "application/json",
      "rel": "actions"
    }
  ],
  "name": "Acme Toaster",
  "properties": {
    "on": {
      "description": "Whether the toaster is currently heating bread",
      "href": "/things/toaster/properties/on",
      "maximum": null,
      "minimum": null,
      "type": "boolean",
      "unit": null
    },
    "timeRemaining": {
      "description": null,
      "href": "/things/toaster/properties/timeRemaining",
      "maximum": null,
      "minimum": null,
      "type": "number",
      "unit": "seconds"
    }
  },
  "type": "Toaster"
}
  • Setting a thing property
$ curl localhost:3000/things/toaster/properties/on -XPUT -d '{"on": "true"}'

{
    "on": "true"
}
  • Getting a thing property
$ curl localhost:3000/things/toaster/properties/on

{
    "on": "true"
}
  • Requesting an action
$ curl localhost:3000/things/toaster/actions -XPOST -d '{"pop": {"input": 10}}'

{
  "pop": {
    "href": "/things/toaster/actions/pop/a4a4aec2-1fa2-4abf-ace7-86bdf8e6e47b",
    "input": 10,
    "status": "pending"
  }
}
  • Getting the actions queue
$ curl localhost:3000/things/toaster/actions

[
  {
    "pop": {
      "href": "/things/toaster/actions/pop/32ddeb33-bdc2-42d2-9bed-c5f5d0a2678a",
      "input": "12",
      "status": "completed",
      "timeCompleted": "2018-04-06 15:21:00.394353",
      "timeRequested": "2018-04-06 15:21:00.382574"
    }
  },
  {
    "pop": {
      "href": "/things/toaster/actions/pop/a4a4aec2-1fa2-4abf-ace7-86bdf8e6e47b",
      "input": "10",
      "status": "completed",
      "timeCompleted": "2018-04-06 15:20:21.675748",
      "timeRequested": "2018-04-06 15:20:21.665374"
    }
  }
]
  • Getting the action status
$ curl localhost:3000/things/toaster/actions/pop/32ddeb33-bdc2-42d2-9bed-c5f5d0a2678a

{
  "pop": {
    "href": "/things/toaster/actions/pop/32ddeb33-bdc2-42d2-9bed-c5f5d0a2678a",
    "input": "12",
    "status": "completed",
    "timeCompleted": "2018-04-06 15:21:00.394353",
    "timeRequested": "2018-04-06 15:21:00.382574"
  }
}
  • Pushing an event
$ curl localhost:3000/things/toaster/events -XPOST -d '{"ready": {"data": true}}'

{
  "ready": {
    "data": true
  }
}
  • Getting the event log
$ curl localhost:3000/things/toaster/events

[
  {
    "ready": {
      "data": "True",
      "timestamp": "2018-04-06 15:23:08.115557"
    }
  }
]

Originally posted on the UnifiedThings blog.

DevStack - OpenStack test environment for the impatient

If you are working with OpenStack and you do not want a full fledged installation of OpenStack for testing, you can check out DevStack. It is a set of scripts which will setup a test OpenStack environment in a few minutes with one simple script on a single node.

In my case, I used a 12G Virtual Machine with 6 CPU cores, 200G disk and Ubuntu 16.04 as the Linux OS.

  • The first step is to add a user with sudo priveleges. I created a user called stack here, added it to the sudoers list and switched to that user shell.
$ sudo useradd -s /bin/bash -d /opt/stack -m stack
$ sudo tee <<<"stack ALL=(ALL) NOPASSWD: ALL" /etc/sudoers
$ sudo su - stack
  • Next download devstack from the git repository.
$ git clone https://git.openstack.org/openstack-dev/devstack
$ cd devstack
  • You now need to add configuration for OpenStack admin password, database password, etc. In the root of the devstack directory create a local.conf file and add the following.
[[local|localrc]]
ADMIN_PASSWORD=admin
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
  • Now you are all set to start the installation. Just run
$ ./stack.sh

You will see a bunch of packages downloading and getting installed. Wait for the process to exit successfully - and yes, you are done with your OpenStack test installation.

You can open the OpenStack Horizon Dashboard at http://devstack_box_url/.

Getting started with libvirt

Very recently I have been working on a platform to orchestrate VM creation and lifecycle management across a cluster of nodes with capacity, metadata, storage and network management. I was exploring stuff like KVM + QEMU and XEN when I read about libvirt and found it perfectly suiting my needs.

To start off with, KVM (Kernel-based Virtual Machine) is a virtualization infrastructure for the Linux kernel that turns it into a hypervisor, a software which enables us to run Virtual Machines on top of a host machine. XEN is another such hypervisor.

libvirt, on the other hand is an abstraction layer on top of the various hypervisor platforms with a nice API available to manage virtualization. We can choose to use any of the hypervisor backends that libvirt supports to create and manage virtual machines. The list of hypervisors supported by libvirt are listed below:

Source: Wikipedia

libvirt has an API in C for development. It also has bindings in other languages like python, perl, ocaml, ruby, java, Go, PHP and C#.

Let’s start off with installing libvirt first. Since the workstations I use are ArchLinux and OSX, I will stick to these two platforms only (the installation instructions for other platforms are easily available).

On ArchLinux, you can install libvirt with a KVM backend by installing the following packages

sudo pacman -S libvirt qemu ebtables dnsmasq bridge-utils openbsd-netcat

By default, KVM is the default driver enabled.

On OSX, you can install libvirt by running

brew install libvirt

To ensure that the libvirt daemon is running, run the following command

sudo systemctl status libvirtd

Once the libvirt daemon is running you can use the command line client virsh which comes included in the libvirt package to connect to the daemon.

Once inside the virsh client, you can run a bunch of commands to consume the API.

  • Getting the hypervisor hostname
virsh # hostname
playstation
  • Getting the node information
virsh # nodeinfo 
CPU model:           x86_64
CPU(s):              4
CPU frequency:       1800 MHz
CPU socket(s):       1
Core(s) per socket:  2
Thread(s) per core:  2
NUMA cell(s):        1
Memory size:         8066144 KiB
  • Getting the version
virsh # version
Compiled against library: libvirt 2.3.0
Using library: libvirt 2.3.0
Using API: QEMU 2.3.0
Running hypervisor: QEMU 2.7.0

You can use the help command to get a list of all available commands.

In the next post in the libvirt series, I’ll start off with programmatically consuming the libvirt APIs.

Have you checked out Jujucharms?

Over the past month, I have been exploring various PaaS providers in the market to study how they package and present their PaaS offerings to the users differently, when someone at work pointed out Jujucharms by Canonical (the same company behind Ubuntu). And taking a first look at it - I immediately liked it, just because of the clear and simple way in which they have presented application and service modelling.

Quoting for their website, “Juju is an application and service modelling tool that enables you to quickly model, configure, deploy and manage applications in the cloud with only a few commands. Use it to deploy hundreds of preconfigured services, OpenStack, or your own code to any public or private cloud.”

You can explore the modelling tool yourself here. As an example, I have a WordPress model setup with the Wordpress nodes scaled to 3, one apache node acting as a reverse proxy to the wordpress nodes, a mysql master database connected to the wordpress nodes and a slave mysql instance.

Wordpress model

The best thing about this is when you deploy, Juju will take care of all the configuration required at the various nodes. (In the above example, the reverse proxy will be configured to point to the wordpress nodes, the slave database will be configured to replicate from the master database, etc.)

Once done with the modelling, you can deploy the models you generated for your applications and services to any of the public cloud platforms supported by Juju.

Public clouds supported by Juju

Back on the Grid

I have been off the blogging / social grid for quite a long time now (the last blog came way back in mid 2014), so I feel it’s about time that I resume this blog. I have initiated this with trashing away Tumblr - my old blog (meh! I know.) and moving the old posts to Jekyll running on Github Pages.

I have been keeping quite busy with work over the past year and a half - from working quite extensively on new cloud platforms spanning over the IaaS, PaaS and Serverless space, to exploring and learning newer design and architectural methodologies, plus tinkering with technologies and frameworks which went into building these distributed and highly scalable systems. Which brings me back here - because I am quite keen to share the stuff which I have learnt during this period building these platforms.

So that said - you can expect a lot more entries from now on :)