Virtualisation on Virtualthoughts

KubeVirt on ARM64 - CDI Workaround

Mon, 07 Apr 2025 00:00:00 +0000

According to the KubeVirt documentation, CDI is not currently supported on ARM64, which is the architecture my Turing RK1 nodes use.

As a workaround, I experimented with writing an image directly to a PVC which can then be cloned/mounted to a KubeVirt VM. This example dd's an ISO image to a PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: fedora-workstation-pvc
spec:
 accessModes:
 - ReadWriteOnce
 resources:
 requests:
 storage: 30Gi
 volumeMode: Block
---
apiVersion: batch/v1
kind: Job
metadata:
 name: upload-fedora-workstation-job
spec:
 template:
 spec:
 containers:
 - name: writer
 image: fedora:latest
 command: ["/bin/bash", "-c"]
 args:
 - |
 set -e
 echo "[1/3] Installing tools..."
 dnf install -y curl xz
 echo "[2/3] Downloading and decompressing Fedora Workstation image..."
 curl -L https://download.fedoraproject.org/pub/fedora/linux/releases/41/Workstation/aarch64/images/Fedora-Workstation-41-1.4.aarch64.raw.xz | xz -d > /tmp/disk.raw
 echo "[3/3] Writing image to PVC block device..."
 dd if=/tmp/disk.raw of=/dev/vda bs=4M status=progress conv=fsync
 echo "Done writing Fedora Workstation image to PVC!"
 volumeDevices:
 - name: disk
 devicePath: /dev/vda
 volumeMounts:
 - name: tmp
 mountPath: /tmp
 securityContext:
 runAsUser: 0
 restartPolicy: Never
 volumes:
 - name: disk
 persistentVolumeClaim:
 claimName: fedora-workstation-pvc
 - name: tmp
 emptyDir: {}

Which can then be mounted to a VM:

Replicating my vSphere network configuration in Openshift Virtualisation

Mon, 05 Feb 2024 00:00:00 +0000

Red Hat Openshift Virtualisation provides a platform for running and managing Virtual Machines alongside Containers using a consistent API. It also provides a mechanism for migrating VMs from platforms such as vSphere.

As I have both environments, I wanted to deploy an Openshift Virtualisation setup that mimics my current vSphere setup so I could migrate Virtual Machines to it.

Existing vSphere Design

Below is a diagram depicting my current vSphere setup. My ESXi hosts are dual-homed with a separation of management (vmkernel) and virtual machine traffic.

Evaluating Harvester in vSphere

Mon, 20 Dec 2021 00:00:00 +0000

Disclaimer - The use of nested virtualisation is not a supported topology

Harvester is an open-source HCI solution aimed at managing Virtual Machines, similar to vSphere and Nutanix, with key differences including (but not limited to):

Fully Open Source
Leveraging Kubernetes-native technologies
Integration with Rancher

Testing/evaluating any hyperconverged solution can be difficult - It usually requires having dedicated hardware as these solutions are designed to work directly on bare metal. However, we can circumvent this by leveraging nested virtualisation - something which may be familiar with a lot of homelabbers (myself included) - which involves using an existing virtualisation solution provision workloads that also leverage virtualisation technology.

Taking a Modular Approach to my Homelab with Pulumi

Wed, 17 Nov 2021 00:00:00 +0000

Architecture

After reviewing the key components of my lab environment, I translated these into the Pulumi stacks as illustrated in the diagram below. Pulumi has a blog post about the benefits of adopting multiple stacks and I found organising my homelab this way enables greater flexibility and organisation. I can also use stacks as a “template” to further build out my lab environment, for example, repeating the “Tools-Cluster” stack to add additional clusters.

Rancher, vSphere Network Protocol Profiles and static IP addresses for k8s nodes [Updated 2023]

Sun, 29 Mar 2020 00:00:00 +0000

Edit: This post has been updated to reflect changes in newer versions of Rancher.

Note: As mentioned by Jonathan in the comments, disabling cloud-init’s initial network configuration is recommended. To do this, create a file:

/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg

To contain:

network: {config: disabled}

In your VM template.

How networking configuration is applied to k8s nodes (or VM’s in general) in on-premises environments is usually achieved by one of two ways - DHCP or static. For some, DHCP is not a popular option and static addresses can be time-consuming to manage, particularly when there’s no IPAM feature in Rancher. In this blog post I go through how to leverage vSphere Network Protocol Profiles in conjunction with Rancher and Cloud-Init to reliably, and predictably apply static IP addresses to deployed nodes.

Container Packet Inspection with NSX-T

Wed, 10 Jul 2019 00:00:00 +0000

For troubleshooting (or just being a bit nosey) we have a number of tools that allow us to inspect the traffic between two endpoints. When it comes to containers however, our approach has to be adjusted slightly. When using NSX-T as a CNI, we have some of these tools available to us out of the box.

App Overview

For this example, I’ve deployed a standard Wordpress deployment consisting of a frontend (web) pod and a backend (DB) pod. The objective is to identify and capture the traffic between these pods.

PKS, Harbor and the importance of container registries

Wed, 27 Feb 2019 00:00:00 +0000

What are container registries and why do we need them?

A lot of the time, particularly when individuals and organisations are evaluating, testing and experimenting with containers they will use public container registries such as Docker Hub. These public registries provide an easy-to-use, simple way to access images. As developers, application owners, system admins etc gain familiarity and experience additional operational considerations need to be explored, such as:

Organisation - How can we organise container images in a meaningful way? Such as by environment state (Prod/Dev/Test) and application type?
RBAC - How can we implement role-based access control to a container registry?
Vulnerability Scanning - How can we scan container images for known vulnerabilities?
Efficiency - How can we centrally manage all our container images and deploy an application from them?
Security - Some images need to kept under lock and key, rather than using an external service like Docker Hub.

Introducing VMware Harbor Registry

VMware Harbor Registry has been designed to address these considerations as enterprise-class container registry solution with integration into PKS. In this post, We’ll have a quick primer on getting up and running with Harbor in PKS and explore some of its features. To begin, we need to download PKS Harbor from the Pivotal site and import it into ops manager.

vRealize Log Insight + PKS Integration

Sat, 02 Feb 2019 00:00:00 +0000

Introduction

In this blog post, we take a look into the integration between PKS and vRealize Log Insight and how this integration benefits the enterprise. As a bit of a recap:

PKS - PKS is a purpose-built enterprise level container solution leveraging the capabilities of Kubernetes, BOSH, VMware NSX-T, Harbour and more to deliver a highly available, highly flexible container runtime that operates on a number of cloud platforms, both private and public, including vSphere, AWS, Azure and GCP.

vSphere and Containers part 1 - VIC (VMware Integrated Containers)

Wed, 28 Mar 2018 00:00:00 +0000

In this multi-part series, we evaluate the options available to vSphere users/customers wishing to deploy a native container service into an existing vSphere environment.

Part 1 - VIC (VMware Integrated Containers).

Part 2 - PKS (Pivotal Container Service).

Why should we care about containers?

Containers change the way we fundamentally look at application deployment and development. There was a huge shift in the way we managed platforms when server virtualisation came around - all of a sudden we had greater levels of flexibility, elasticity and redundancy compared to physical implementations. Consequently, the way in which applications were developed and deployed changed. And here we are again, with the next step of innovation using technology that is making rifts in the industry, changing the way consume resources.

vRealize Log insight – Frequently Overlooked Centralised Log Management

Mon, 15 Jan 2018 00:00:00 +0000

Log analysis has always been a standardised practice for activities such as root cause analysis or advanced troubleshooting. However, ingesting and analysing these logs from different devices, types, locations and formats can be a challenge. In this post, we have a look at vRealize Log Insight and what it can deliver.

What is it?

vRealize Log Insight is a product in the vRealize suite specifically designed for heterogeneous and scalable log management across physical, virtual and cloud-based environments. It is designed to be agnostic across what it can ingest logs from and is therefore valid candidate in a lot of deployments.

Homelab Networking Refresh

Mon, 18 Dec 2017 00:00:00 +0000

Adios, Netgear router

In hindsight, I shouldn’t have bought a Netgear D7000 router. The reviews were good but after about 6 months of ownership, it decided to exhibit some pretty awful symptoms. One of which was completely and indiscriminately drop all wireless clients regardless of device type, range, band or frequency it resided on. A reconnect to the wireless network would prompt the passphrase again, weirdly. Even after putting in the passphrase (again) it wouldn’t connect. The only way to rectify this was to physically reboot the router.

Embracing the SDDC with NSX-V automation

Mon, 11 Dec 2017 00:00:00 +0000

The Software Defined Data Center (SDDC for short) has become a widely adopted and embraced model for modern datacentre implementations. Conveying the benefits of the SDDC, particularly the non-technical aspects can be a challenge. In this blog post we take a practical example of a single activity we can automate in NSX and the benefits that come from it, both technical and non-technical.

The NSX API

An API (Application Programming Interface), in simple terms is an intermediary that allows two applications to communicate with each other via a common syntax. Although we may not be aware of it, it’s likely we use API’s every day when we use applications such as Facebook, LinkedIn, vSphere and countless others. For example, when you create a logical switch in the vSphere web client, behind the scenes an API call is made to the NSX manager to facilitate that request.

VMware Cloud on AWS

Fri, 24 Nov 2017 00:00:00 +0000

Perhaps one of VMware’s most significant announcements made in recent times is the partnership with Amazon Web Services (AWS), including the ability to leverage AWS’s infrastructure to provision vSphere managed resources. What exactly does this mean and what benefits could this bring to the enterprise?

Collaboration of Two Giants

To understand and appreciate the significance of this partnership we must acknowledge the position and perspective of each.

Market leader in private cloud offerings
Deep roots and history in virtualisation
Expanding portfolio

Homelab - Nested ESXi with NSX and vSAN

Tue, 03 Oct 2017 00:00:00 +0000

The Rebuild

I decided to trash and rebuild my nested homelab to include both NSX and vSAN. When I attempted to prepare the hosts for NSX I received the following message:

I’ve not had this issue before so I conducted some research. I found a lot of blog posts / comments / KB articles linking this issue to VUM. For example : https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2053782

However, after following the instructions I couldn’t set the “bypassVumEnabled” setting. Nor could I manually install the NSX vibs and was presented with the following:

vDS to vSS and back again

Sun, 02 Jul 2017 00:00:00 +0000

Overview

I was recently tasked with migrating a selection of ESXi 5.5 hosts into a new vSphere 6.5 environment. These hosts leveraged Fibre Channel HBA’s for block storage and 2x10Gbe interfaces for all other traffic types. I assumed that doing a vDS detach and resync was not the correct approach to do this, even though some people reported success doing it this way. The /r/vmware Reddit community agreed and later I found a VMware KB article that backs the more widely accepted solution involving moving everything to a vSphere Standard Switch first.

Intel Skylake/Kaby Lake processors: broken hyper-threading

Mon, 26 Jun 2017 00:00:00 +0000

Overview

Source : https://lists.debian.org/debian-devel/2017/06/msg00308.html

It appears some Intel Xeon CPU’s are susceptible to a recently discovered Hyper Threading bug. However, these are limited to E3 v5/v6 based Xeon systems which are found mostly in entry level servers with single socket implementations. > Dual socket systems currently leverage E5 based Xeons which don’t appear to be affected.

Currently, the easiest way to mitigate against this bug is to simply disable hyper-threading. The bug also appears to be OS agnostic.

Homelab v2 - Part 1

Fri, 19 May 2017 00:00:00 +0000

Out with the old

My previous homelab, although functional was starting to hit the limits of 32GB of RAM, particularly when running vCenter, vSAN, NSX, etc concurrently.

A family member had use for my old lab so I decided to sell it and get a replacement whitebox.

Requirements

Quiet - As this would live in my office and powered on pretty much 24/7 it need a silent running machine
Power efficient - I’d rather not rack up the electric bill.
64GB Ram Support

My Nested NSX Home Lab

Sun, 15 Jan 2017 00:00:00 +0000

With the ever growing popularity of SDDC solutions I’ve decided to invest some time in learning VMware NSX and sit the VCP6-NV Exam. For this I’ve re-purposed my existing homelab and configured it for NSX. I have a fairly simple setup consisting of a single whitebox “server” that will accommodate nested ESXi hypervisors and a HP Microserver acting as a iSCSI target.

Whitebox specs:

Motherboard: MSI B85M-E45 Socket 1150

CPU: Intel Core i7 4785T 35W TDP

VCP6-NV Study Materials

Fri, 18 Nov 2016 00:00:00 +0000

NSX is a very exciting technology and I’ve made it a personal goal to sit and (hopefully) pass the VCP6-NV Exam. I hear it’s a tough exam, so it should provide a good challenge.

In preparation for this my list of resources are as follows:

Plenty of experimenting in my homelab
VCP6-NV Official Cert guide from Amazon.co.uk
VMware NSX for vSphere Network Virtualization Design Guide

More resources will be added as I find them. To anyone else wishing to pass this exam - Good luck!

An introduction to vSphere Metro Storage Cluster with Compellent Live Volume

Thu, 16 Jun 2016 00:00:00 +0000

Intro

VMware vSphere Metro Storage Cluster is a suite of infrastructure configurations that facilitate a stretched cluster setup. It’s not a feature like HA/DRS that we can switch on easily; it requires architectural design decisions that specifically contribute to this configuration. The foundation of which are stretched clusters and with regards to the Compellent suite of solutions Live Volume

Stretched Cluster

Stretched clusters are pretty much self explanatory. In comparison to a lot of configurations where compute clusters reside within the same physical room, stretched clusters spread the compute capacity over more than one physical location. This can still be internally (different server rooms within the same building) or further apart over geographically disperse sites.

Facilitating multi-SAN Environments

Mon, 23 May 2016 00:00:00 +0000

The what, where and why

At the moment I’m involved in a fair amount of project work involving data (VM) migrations between different storage platforms, sometimes even from the same vendor. What seems to be quite a simple process can get complicated depending on the design considerations for both storage platforms and the migration criteria - particularly if both storage platforms need to co-exist for either the short or long term and migrated VM’s must be performed live.

In-guest iSCSI to native VMDK

Thu, 05 May 2016 00:00:00 +0000

Do we really need in-guest iSCSI volumes?

Well, yes and no.

I’ll admit, the need for VM’s with their own iSCSI initiator have decreased over the various improvements made to vSphere and ESXi. However, I would imagine there are a number of implementations that (justifiably) still need this arrangement, and those that don’t. I was recently tasked with eliminating a number of guest-initiated iSCSI disks in favor of using native VMDK’s.

VCAP5-DCD Experience

Mon, 02 May 2016 00:00:00 +0000

Motivation

Towards to end of 2015 I decided to make passing the VCAP5-DCD exam a target. I had already refreshed my VCP certification to version 6 and wanted to move further up VMware’s certification chain. It may have seen unusual timing giving the impending release of version 6 of the VCAP exams, but as I was in between jobs at the time it seemed like a good idea to not only demonstrate some additional skills, but enhance my CV for potential employers.

VCP6-NV Passed!

Sun, 01 May 2016 00:00:00 +0000

Motivation

SDDC solutions are becoming increasingly more popular, and although I’m probably a little biased, I would say that NSX is leading the software defined networking front. Following on from my previous post about my nested NSX homelab I sat and (thankfully passed) the VCP6-NV exam.

Study Materials

The official cert guide - https://www.amazon.co.uk/VCP6-NV-Official-2V0-641-Vmware-Certification/dp/0789754800/ref=sr_1_1?ie=UTF8&qid=1493819639&sr=8-1&keywords=vcp6-NV

Practice (non exam) questions - http://www.elasticsky.co.uk/practice-questions/

Blueprint - https://mylearn.vmware.com/mgrReg/plan.cfm?plan=95141&ui=www_cert

Pluralsight NSX videos

Homelab

Lots of time

Experience

This was a particularly tough exam. As I would describe myself as somewhat inexperienced in NSX compared to vSphere, I found the exam challenging. I’ve noticed that VMware have recently revised the exam reducing the number of questions down to 77 from 85, which is nice.