BOSH is a powerful tools in provisioning and deploying software over VMs consistently. BOSH will perform monitoring, failure recovery and software updates with zero-to-minimal downtime. With BOSH, application platform like VMware Tanzu Application Service (formerly Pivotal Platform Application Service) and Pivotal Container Service (PKS) can be deployed in a consistent manner.
However, sometimes there might be custom OS configuration required. BOSH os-conf release is developed to enable configuration of Linux OS, for example:
- Customize login banner text
- Add UNIX users to VM
- Add system wide CA Certificate
- Configure resolv.conf search domain
- Change TCP keepalive kernel args
- Apply arbitrary sysctls
In this example, we will create a custom SSH login user for Master and Worker nodes in BOSH deployed PKS cluster. There are various use case that can utilize os-conf BOSH release including creating temporary service account for vulnerable scanning (VA scanning) or anti-virus agent and applying custom login banner to OS.
Steps to Enable Custom SSH Login User
Steps for creating a custom SSH login for BOSH-deployed VMs is defined as below:-
- Download os-conf BOSH release into jumphost (Pivotal Operation Manager in this guide)
- Create os-conf BOSH release tarball
- Upload BOSH release tarball to BOSH director
- Create SSH key pair for VMs access
- Create runtime-config that fixed to BOSH deployment
- Run update runtime-config on BOSH
Download os-conf BOSH release
Release and source code for os-conf BOSH release can be obtained at cloudfoundry/os-conf-release. First, in jumphost (Pivotal Operation Manager in this example), we will clone os-conf source code into OpsMan directory and navigate to the directory of os-conf-release.
git clone https://github.com/cloudfoundry/os-conf-release.git cd os-conf-release
Create os-conf BOSH release tarball
Next, we will package and create the os-conf BOSH release tarball in jumphost based on the required version of release (version 21.0.0 in this example).
While creating BOSH release, some additional packages is needed to be downloaded from internet. Do run the
bosh create-release command on jumphost that has internet access. Result of the command line will be os-conf release tarball created in .tgz format.
bosh create-release releases/os-conf/os-conf-<VERSION>.yml --tarball os-conf-<VERSION>.tgz
Upload Tarball Release to BOSH Director
Tarball for os-conf release created in previous step will then be uploaded to BOSH director using
bosh upload-release command. Wait for BOSH director to extract, verify and create new release. Verify that os-conf release has been uploaded to BOSH director using
bosh releases command. Ensure the correct version uploaded to BOSH director.
bosh -e <BOSH_ENV> upload-release os-conf-<VERSION>.tgz bosh -e <BOSH_ENV> releases
Generate SSH Key Pair
Adding OS user to BOSH-deployed VMs required credentials to be created in the first place. In os-conf BOSH release, credentials can be added to user is two ways:-
- Username with Password
- Username with SSH private and public key pair
In this example, we will be adding user to BOSH-deployed VMs using username with SSH private and public key pair which is a more secure way of passing OS credentials. We will create the SSH private and public key pair in this step by using
ssh-keygen command. Note that other method of generating SSH key like
PuTTYgen will also work.
Create Runtime Config YAML File
Runtime config is a way that BOSH used to update or define IaaS agnostic configuration that applies to deployments. Based on created runtime config YAML file, BOSH director will apply the changes to specified deployment during the next
bosh deploy cycle.
Addons block in runtime config can be used to specify placement rules for required BOSH-deployed VMs. Placement rules is defined using
include (inclusion placement rules) or
exclude (exclusion placement rules). Available rules for the addons directives are:
- os – Based on stemcell’s operating system
- deployments – Based on deployment names
- name – Based on job names
- release – Based on job release names
- instance_groups – Based on instance group names
- lifecycle – Based on lifecycle type, either
- networks – Based on network names
- teams – Based on team names
Reference: Placement rules for BOSH runtime-config
releases: - name: os-conf # Release name version: 21.0.0 # Release version addons: - name: os-conf jobs: - name: user_add release: os-conf properties: persistent_homes: true # If set to true, user home directories are created on persistent storage in /var/vcap/store/home users: - name: va-scan public_key: "<PUBLIC_SSH_KEY>" # Add in Public SSH key created shell: /bin/bash # OPTIONAL: Defaults to `/bin/bash` sudo: true # OPTIONAL: Defaults to `true` include: deployments: [<DEPLOYMENT_ID>] # Add in Deployment ID obtain using bosh deployments command
Update Runtime Config in BOSH Director
After compiling of the runtime config YAML file with parameters suited to use case (applied to one PKS cluster deployment in this example), run
bosh update-runtime-config command to apply runtime config to BOSH director. Run
bosh configs to verify if runtime config updated in BOSH director.
bosh -e <BOSH_ENV> update-runtime-config --name=create_ssh_user ./runtime-config.yml bosh -e <BOSH_ENV> configs
Once verified runtime config updated in BOSH director, apply changes to deployment by running
apply changes in Pivotal Operation Manager (OpsMan) or trigger update of PKS cluster using
pks update-cluster command. On OpsMan, run
bosh tasks to check status of rolling update on deployment and run
bosh vms to check on status of deployment once rolling update of VMs completed by BOSH director.
pks login -a <PKS_API_FQDN> -u <USERNAME> -p <PASSWORD> pks update-cluster <CLUSTER_NAME> --num-nodes <NUMBER_OF_NODES> bosh -e <BOSH_ENV> task <TASK_ID> bosh -e <BOSH_ENV> -d <DEPLOYMENY_ID> vms
After rolling update on BOSH-deployed VMs (which are PKS nodes in this example) completed, BOSH-deployed VMs can be access using SSH username and SSH private key created in previous step.
ssh -i <PRIVATE_KEY_FILE_NAME> <USER_NAME>@<VM_IP>
With this, we have successfully apply os-conf BOSH release to the deployment. External user will now be able to SSH access to BOSH-deployed VMs using SSH private key and/or password created and defined.
- BOSH os-conf Release Usage – https://bosh.io/releases/github.com/cloudfoundry/os-conf-release?version=22.0.0
- BOSH os-conf GitHub Link –https://github.com/cloudfoundry/os-conf-release/tree/ef7220d/
- BOSH Director Runtime Config Usage – https://bosh.io/docs/runtime-config/
- Generating SSH Key using SSH Keygen – https://www.ssh.com/ssh/keygen/