Using Helm to Deploy Blockchain to Kubernetes
Background
is an online travel agency responsible for managing thousands of travel bookings daily in Australia and New Zealand. Over the last year, Microsoft and Webjet have collaborated to help Webjet solve inefficiencies in the processing and management of hotel bookings by leveraging blockchain technology. You can learn more about our partnership from the . For more technical details watch this .
As Webjet continued to grow their blockchain deployment, they began to hit several issues with their Infrastructure-as-a-Service (IaaS) architecture. We recently worked with Webjet to help migrate their deployment to . In this code story, we’ll share the lessons learned from migrating from IaaS to Kubernetes, explain how we used Helm to deploy a private network to Kubernetes, and show how you can use Helm to deploy your own private Ethereum network.
The Road to Kubernetes
Due to (VMSS) feature of , Webjet chose to host their private Ethereum networks on a series of VMSS with each one running Docker containers orchestrated by . A generalized version of Webjet’s Azure deployment is available on . However, in Webjet’s path to moving this to production, some deficiencies in their IaaS-based architecture became apparent.
Difficult to scale
Docker Compose supports running multiple for a container; however, in order to load balance across replicas, management of container-host port mappings is required.
At the node-level, as VMSS’ autoscale feature scales out the entire node, VMSS work optimally against homogeneous workloads. With several different services running on the machine, scaling up/down a VMSS would affect all running services on that machine. Alternatively, the architecture could be separated into several Docker Compose files and thereby several VMSS, but this would require a significant amount of pre-planning to ensure a proper balance of hardware needs and costs.
Error-prone upgrades
Upgrading services meant SSH-ing into the VMs, running docker-compose stop, downloading the new Docker Compose manifest, and then subsequently running docker-compose up. Configuration management tools such as could be used to automate this process, but the process itself is very custom and has a high potential for error.
For the above reasons, the Webjet team decided to leverage Kubernetes through to help automate deployment, scaling, and orchestration of the application. With Kubernetes, Webjet is able to scale up/down certain applications and take advantage of Kubernetes’ native support for rolling upgrades.
Architecture
After migrating their existing Docker Compose manifests to Kubernetes, we landed on the following architecture:
The diagram above shows the architecture of the private Ethereum network when mapped to Kubernetes constructs. The blue represents Kubernetes , the red represents Kubernetes , whereas the green and yellow are Kubernetes and respectively.
Every time Webjet needed to deploy a new Ethereum network, a series of configuration updates needed to be made to the YAML definitions. These changes included the Ethereum genesis file, application secrets, and Geth private keys. Webjet initially accomplished this task using shell scripts that would search and replace configuration values and then execute the deployment using kubectl create. During a hackfest with Webjet, we worked together to instead use Helm to manage the templatization of the YAML definitions.
Helm
is a package manager for Kubernetes. The project was initially created by and has since been donated to the . Sidenote, the lovely folks of Deis are now . Typically, an application is composed of a combination of services, deployments, secrets, etc. Instead of managing these Kubernetes resources individually, Helm offers a higher-level construct (known as charts) to manage your entire application. With Helm, you can create, upgrade, and rollback entire applications, and easily share applications/charts with peers and the greater community.
Creating a Chart
Working with Webjet, we created a Helm chart to deploy a private Ethereum network. To create a Helm chart, install the , and run helm create to scaffold a new chart. To migrate your existing YAML manifests, copy the manifests to the Helm folder structure and add the following labels outlined here: These labels are not required to deploy charts but are recommended for consistency. The next step is to templatize configurations and settings that can be overridden. For more details, see the .
With Helm, each chart is an individually-manageable unit. Webjet separated their architecture into multiple Helm charts such that they could be individually upgradeable. For instance, we had separate charts for the Ethereum network (Miners, EthStats, Bootnode), another for the Blockchain Watcher, one for deploying their Storage (SQL), etc. Artifacts that were shared across multiple Helm charts (e.g., Secrets, ConfigMaps) were marked as .
Deploying a Chart
Helm is comprised of two components: a client CLI () and a server (). One of the niceties of deploying a Kubernetes cluster on Azure through either or , is that by default the cluster will be pre-provisioned with Tiller. For a cluster that has not yet been initialized, you can do so with helm init.
To install a chart, you can run helm install, which will search for and install a chart from the . The Ethereum chart we developed in concert with Webjet is available on GitHub, and once is merged, you’ll be able to install a private Ethereum network onto your Kubernetes cluster using:
| 1 | helm install incubator/ethereum |
Until the pull request is merged, I have made available the Chart archive:
| 1 2 3 4 | helm install https://japoonhelmstrg.blob.core.windows.net/public/ethereum-0.1.0.tgz --set geth.account.publicKey=[PUBLIC_KEY] --set <span class="blob-code-inner annotated">geth.account.privateKey</span>=[PRIVATE_KEY] --set <span class="blob-code-inner annotated">geth.account.<span class="">secret</span></span>=[SECRET] |
Note that geth.account.publicKey, geth.account.privateKey, and geth.account.secret are required configurations. To create a new Geth account, please refer to .
You now have a private Ethereum network running inside your Kubernetes cluster.
To clarify, this chart deploys a private Ethereum network that is not connected to MainNet.
With Webjet, their charts were committed to a Git repository and all deployments were initiated with Helm by supplying a local filepath to the Chart. However, a private Helm can be easily provisioned to host your private Charts.
Summary
This code story explains how Webjet migrated their deployment to Kubernetes and shows how Helm can be applied to simplify the deployment and management of applications hosted on Kubernetes. Using Kubernetes on Azure has greatly simplified Webjet’s deployment process. Our solution using Helm removed the need for Webjet to manage their own deployment scripts and makes it possible to leverage the many existing charts made available by the community.
Charts can be submitted to the Charts registry and shared with the community. The Ethereum chart shown in the article is open-sourced, and there is an to add it to the Charts registry. This Chart deploys a simplified private Ethereum network; in order to deploy a network in a more highly available manner, please refer to . If you have any questions or feedback on this code story, please reach out in the comments below.