docs-onboarding/vms-and-lxcs.md

# Basic Know-How about VMs and LXC containers

## Microsft Azure compatible Linux Distributions

We love Debian, but it's not well supported by Microsoft Azure. Thus we are once in a while going mainstream and use Ubuntu, if VMs must be connected to Azure.

## User Management

- Limit root login to console only as an emergency fallback. You can login via the proxmox console then.
- Create a `ansible` super user with sudo rights; allow SSH access by keys only. This is used for maintenance and configuration.
- Create a normal user `debian` with restricted privileges; also allow SSH by keys only. This one can be used for normal system tasks.
- Disallow password-based SSH logins for all users besides root.
- Periodically review SSH logs for unauthorized access attempts.

## IaC vs. CaC

Infratsructure as Code (IaC) is how we deploy virtual bare-metal. We are using Terraform for that.

Configuration as Code (CaC) is how we configure the VMs and LXC containers and install software. We are using Ansible for that.

We are defining the boundary between IaC and CaC as follows:

- IaC is responsible for the VMs and LXC containers, the network, and the storage.
- IaC ends as soon as the VMs and LXC containers are up and running.
- SSH keys are installed by IaC.
- CaC is responsible for the software installed on the VMs and LXC containers.
- CaC uses the SSH keys installed by IaC to connect to the VMs and LXC containers.

## IaC Terraform Proxmox Provider

The [Proxmox Terraform Provider](https://github.com/Telmate/terraform-provider-proxmox) is not mature enough now. Thus we use [Proxmox VE Helper-Scripts](https://community-scripts.github.io/ProxmoxVE/scripts).

The following text in this chapter are notes and references in case the provider gets more mature and we switch in the future.

Our hypervisor is Proxmox, which is based on Debian. We are using the [Proxmox cloud-init](https://pve.proxmox.com/wiki/Cloud-Init_Support) template for Ubuntu.

We are using a small server images to keep the attack surface small. The cloud-init template is a server Ubuntu image with cloud-init installed. Get the URL from the Ubuntu website and download it to the Proxmox servers local storage for ISO images. Ubuntu website link: [https://cloud-images.ubuntu.com/releases/](https://cloud-images.ubuntu.com/releases/). We are going for the file ending in `*server-cloudimg-amd64.img`.

We are also using Ubuntu for the LXC containers. We are using the latest Ubuntu standard LXC template you can download via the Proxmox web interface for that.

## SSH keys

SSH keys are managed via the approach described in the [infra-terraform-sshkeyvault](https://xwr.visualstudio.com/jambor.pro/_git/infra-terraform-sshkeyvault) repository. As of now we create them one by one with the provided scripts.

## Create an LXC container

- Make use of the [Azure Naming Tool](https://app-azurenamingtool-dev-bnhfgbdgafeqh2gf.switzerlandnorth-01.azurewebsites.net/) to get a suitable name for the LXC container. We use the same schema as for virtual machines. E.g. `vm-mal-dev-opr-1`
- Create a new ssh key according to the [infra-terraform-sshkeyvault](https://xwr.visualstudio.com/jambor.pro/_git/infra-terraform-sshkeyvault) repository. Use a name from the naming tool, e.g. `kvs-mal-dev-opr-1`

- Search for a pre-defined template or the latest Debian / Ubuntu empty template: [Proxmox VE Helper-Scripts](https://community-scripts.github.io/ProxmoxVE/scripts)
- Review the script and check that you understand it and no malicious code is in it. (ha ha, we all do that, right?)
- Execute the script on the Proxmox servers shell via the web interface. SSH is not advised for that.
- Use advanced settings like the example below.

```bash
  🧩  Using Advanced Settings on node prd-proxmox-2
  🖥️  Operating System: debian
  🌟  Version: 12
  📦  Container Type: Unprivileged
  🔐  Root Password: ********
  🆔  Container ID: 101
  🏠  Hostname: vm-mal-dev-opr-1
  💾  Disk Size: 64 GB
  🧠  CPU Cores: 1
  🛠️  RAM Size: 2048 MiB
  🌉  Bridge: vmbr0
  📡  IP Address: dhcp
  🌐  Gateway IP Address: Default
  📡  APT-Cacher IP Address: Default
  🚫  Disable IPv6: yes
  ⚙️  Interface MTU Size: Default
  🔍  DNS Search Domain: Host
  📡  DNS Server IP Address: Host
  🏷️  Vlan: 7
  📡  Tags: ;
  🔑  Root SSH Access: yes
  🔍  Verbose Mode: yes
```

- **Important:** add the public ssh key to the LXC in the process to enable ssh via key.
- If the service is exposing an http(s) service, put traefik infront of it if you want to access it from external. See [Proxmox VE Helper-Scripts](https://community-scripts.github.io/ProxmoxVE/scripts) for examples.

If you cannot choose Ubuntu as distribution, and you must connect the VM to Azure you should choose to create an empty Ubunto LXC and install the desired service on top of that.

- Create a LXC within the Proxmox web interface and use the latest Ubuntu LTS template.
- **Important networking note** using IPv6 dhcp causes the network to stop working as the lease seems not to be updated. Keep IPv6 as static, IPv4 can be dhcp.
- Ensure to set the right vnet ID according to [networking instructions](network.md).
- ssh into the LXC container making use of the ssh key.
- Install waht ever you need to install. Preferably use Ansible for that.

## Create a VM

- ...

## Add new resource to Ansible repository

We are maintaining VMs and LXCs with Ansible. Add the newly created VM or LXC to the Ansible inventory [infra-ansible-serverconfiguration](https://xwr.visualstudio.com/jambor.pro/_git/infra-ansible-serverconfiguration).