Private AKS Access

Introduction

In the Private VS Public AKS security concerns – fredrkl blog post, we examined the difference between public and private AKS API servers. Let's say you have made the security considerations and decided to use a private API server in production. This blog post will look at setting up an AKS cluster with the VNetintegrations feature, enabling us to turn on or off the API servers’ public API. This is convenient in scenarios where we may want a public cluster in a non-production environment but secure the cluster even more in production, having it private.

AKS VNet integration

The code for this blog post is located at: https://github.com/fredrkl/private-aks-demo. If we take a look at the VNet setup, we have a separate subnet for the K8s API control plane. This is where AKS will place its API server. Also, we need to set the subnet with the service_delegation, allowing AKS to take charge of that subnet and make any necessary changes. For example, The security group can limit traffic between Jumphost and AKS nodes. The VNet is in effect like this:

Warning: This example is one way of setting up secure access to the AKS API server. You might want to split out the jump host and bastion to a separate VNet altogether, do a VNet link, or maybe centralize a single jump host. Simple is sometimes more secure, and we will, for now, go for this single VNet with multiple subnets.

It is impossible to restrict API server access through whitelisting if it is in private mode. If you want to restrict traffic, you must use Network Security Groups, which makes sense.

Bastion

Azure Bastion is a fully managed Platform as a Service (PaaS) offered by Microsoft, providing secure and seamless Remote Desktop Protocol (RDP) and SSH connectivity to virtual machines directly in the Azure portal or from your local machine. This service ensures that your virtual machines do not need to be exposed to the public internet. Furthermore, accessing a VM or jump host through the web browser prevents you from having your local machine as a connected device in security settings, e.g., PCI-DSS.

Connecting to the jump host from your local computer through the bastion host using SSH or Azure Entra ID is possible. Please see the official documentation at: https://learn.microsoft.com/nb-no/azure/bastion/connect-vm-native-client-linux.

The Azure Bastion does not function as a jump-host itself. Instead, it provides a secure and private RDP/SSH connection to your virtual machines (VMs) in Azure. The VMs essentially act as the jump-hosts, providing a secure environment for you to execute tasks without exposing your VMs to the public internet.

Linux Jump Host

To set up a Linux VM as a jump host in Terraform, you must define a resource block for the VM. Below is a brief example:

resource "azurerm_linux_virtual_machine" "jumphost" {
  name                            = "jumphost"
  resource_group_name             = azurerm_resource_group.example.name
  location                        = azurerm_resource_group.example.location
  size                            = "Standard_D2s_v3"
  admin_username                  = "adminuser"
  network_interface_id            = azurerm_network_interface.example.id

  admin_ssh_key {
    username   = "adminuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "22.04-LTS"
    version   = "latest"
  }
}

This script creates an Ubuntu 22.04-LTS LTS VM in your specified Azure resource group. The VM size is set to Standard_D2s_v3 and uses an existing network interface. It also sets up SSH access for the admin user using a public key stored in your local .ssh directory.

Gotchas

• The VM, when using Linux, at the time of this writing needs to be a Linux server
• The SSH admin username needs to be exactly adminuser
• For Azure AD login to work, the Ubuntu version must be newer than 18.04

Azure AD login

If we want to enable Azure AD login to the VM, then we need to extend the script with an add-on:

resource "azurerm_virtual_machine_extension" "AADSSHLoginForLinux" {
  name                       = "AADSSHLoginForLinux"
  virtual_machine_id         = azurerm_linux_virtual_machine.jumphost.id
  publisher                  = "Microsoft.Azure.ActiveDirectory"
  type                       = "AADSSHLoginForLinux"
  type_handler_version       = "1.0"
  auto_upgrade_minor_version = true
}

Then assign your AD user directly or through a group access to the VM with either:

• Virtual Machine Administrator Login
• Virtual Machine User Login

resource "azurerm_role_assignment" "example" {
  scope              = azurerm_linux_virtual_machine.example.id
  role_definition_id = data.azurerm_role_definition.vm_admin.id
  principal_id       = data.azuread_group.admins.object_id
}

data "azurerm_role_definition" "vm_admin" {
  name = "Virtual Machine Administrator Login"
}

data "azuread_group" "admins" {
  object_id = "43f5e30f-0e58-47a1-93e0-7e8342f890b0"
}

You should now be able to log in using your Azure AD account:

az network bastion ssh --name "<BastionName>" --resource-group "<ResourceGroupName>" --target-resource-id "<VMResourceId or VMSSInstanceResourceId>" --auth-type "AAD"

Conclusion

In conclusion, leveraging Azure Bastion to access Azure Kubernetes Service (AKS) presents a highly secure and manageable approach to orchestrating and accessing containerized applications. With this setup, organizations can significantly enhance the security of their AKS clusters by eliminating the need for direct public internet exposure. The seamless RDP and SSH connectivity offered by Azure Bastion ensures that the integrity and confidentiality of data and applications are preserved. This blog post presented a step-by-step setup guide for Azure Bastion and its integration with AKS, aiming to serve as a valuable resource for organizations seeking to bolster their cloud security infrastructure.