This is a guide for setting up Cloud Foundry in a hybrid environment with virtualization and bare-metal infrastructure. Cloud Foundry is made up of many components. It makes sense to run most of them in a virtual environment for resource sharing and separation reasons. And these pieces might not require the resources for an entire physical machine all to itself.

Computing components, such as DEA Runner or Diego Cells, should run on Bare-Metal to fully make use of the computing power, without spending resources for your virtualization tier.

This tutorial will guide you step by step, to set up Cloud Foundry components on vSphere
and DEA Runner on Bare-Metal machines. When you finish, your environment will look like this:

Pre-Requisites

Before we begin, your environment should look like this:

You should have:

1. One or more bare-metal machine(s)
2. vSphere environment with vCenter installed
3. Two networks set up in vSphere: private and public
4. The Bare-Metal machine(s) should be on the private network

Steps to install Hybrid Cloud Foundry:

1. Install Bosh Director with vSphere CPI

To install Cloud Foundry on vSphere, we need to first deploy a Bosh Director with the vSphere CPI. Follow this link to initialize a Bosh director in your vSphere environment. After the installation, your environment should look like this:

2. Install Cloud Foundry on vSphere

Now that the vSphere Bosh Director is installed, we are to deploy Cloud Foundry in our vSphere environment. You can follow these steps to deploy Cloud Foundry. That might not always be easy. If you run into a wall, you can change the minimal-aws.yml manifest to fit in your own environment. I have found using the example manifest to be a lot easier than doing the full blown stub and spiff approach, but please know that this environment is not production ready. My Cloud Foundry deployment manifest can be found here.

After the deployment, your
bosh vms
output should look like the following:

At this time, there should be 13 Cloud Foundry VMs, including 1 Runner.

Now, verify that Cloud Foundry is actually working. I used the dojosnake game to test my CF.

You can test it out by doing the following:

At this point, the dojosnake should be up and running and you can point your browser to dojo-snake.{YOUR APP DOMAIN}.

Great! At this point, you should have a functional Cloud Foundry on your vSphere!

1. Install RackHD
   Please reference RackHD documentation: http://rackhd.readthedocs.io/en/latest/tutorials/index.html.

At this point, you should have a functional RackHD. Your environment should look like this:

4. Install Bosh Director with RackHD CPI

Now that we have a functional RackHD server, we are ready to install a Bosh Director with RackHD CPI. What’s interesting is that now we can have Bosh Director deployed as a vSphere VM in the vSphere environment. We would not need to use

to accomplish this. We can just use the vSphere Bosh Director to deploy the RackHD Bosh Director.

In order to do that, you can do the following:
1. bosh target {{REPLACE_WITH_VSPHERE_BOSH_IP}}
2. Download Bosh Release
3. bosh upload release
4. git clone https://github.com/cloudfoundry-incubator/bosh-rackhd-cpi-release.git
5. cd bosh-rackhd-cpi-release
6. bosh create release
7. bosh upload release
You’ll need a manifest for the bosh deployment. Feel free to use this example manifest and fill in the blanks.

After getting the manifest ready, you can just run   bosh deploy!

At this point, your environment should look like this:

5. Deploying Runners to Bare Metal machines!

If you have made it this far, then deploying runners to Bare-Metal machines should be a breeze.

First, we’ll remove the runner from the vSphere environment. That can easily be done with Bosh by setting the instance count of runner to 0 in your Cloud Foundry deployment manifest:

Then run

again. Bosh will remove the runner from the vSphere environment for you. At this point, your applications running on Cloud Foundry should no longer work! If the application, like dojosnake, is still working, then something is wrong!

We’ll need to do the following:
1.

1. Download OpenStack KVM (raw) stemcell from bosh.io

4.

5.

At this point, you can just duplicate your Cloud Foundry deployment manifest and remove everything, leaving only the runner job. Alternatively, you can also reference this example manifest.

Now you are ready to

With some luck, your deployment would look something like this:

Your environment should look like this:

Now that the runner is up and running, you should be able to restart your applications in Cloud Foundry and see them working now. Feel free to scale out your runner into more machines if needed!

Reach out to me with any feedback or questions! I usually hang out in the cloudfoundry.slack.com   #bosh-rackhd-cpi channel. I can also be reached at victor.fong@emc.com or on twitter @victorkfong.

Tags: bare-metal, baremetal, Bosh, cloud foundry, cloudfoundry, cpi, DEA, deploy, diego, manifest, rackhd, runner, vsphere