Update README.md
This commit is contained in:
parent
c10bb3794a
commit
629061bc09
1 changed files with 132 additions and 106 deletions
238
README.md
238
README.md
|
|
@ -4,16 +4,15 @@
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
|
||||||
## Demo Video
|
## Demo Video
|
||||||
<video controls src="https://mentalnet.xyz/forgejo/markmental/mnmivm-se/raw/branch/master/assets/output.mp4" title="Video Title">
|
<video controls src="https://mentalnet.xyz/forgejo/markmental/mnmivm-se/raw/branch/master/assets/output.mp4">
|
||||||
Your browser does not support the video tag. Video playing is in /assets/output.mp4
|
Your browser does not support the video tag. Video is available in /assets/output.mp4
|
||||||
</video>
|
</video>
|
||||||
|
|
||||||
|
**MNMIVM-SE** is the **server-focused edition** of MNMIVM
|
||||||
|
(https://mentalnet.xyz/forgejo/markmental/mnmivm-se) — a minimal, single-binary VM launcher built on **QEMU + KVM + cloud-init** that turns your **LAN into a local VM cloud**.
|
||||||
|
|
||||||
**MNMIVM-SE** is the **server-focused edition** of MNMIVM (https://mentalnet.xyz/forgejo/markmental/mnmivm) — a minimal, single-binary VM launcher built on **QEMU + KVM + cloud-init** that turns your **LAN into a local VM cloud**.
|
Unlike traditional platforms, MNMIVM-SE exposes raw infrastructure primitives directly:
|
||||||
|
|
||||||
Unlike traditional platforms, MNMIVM-SE exposes the raw infrastructure primitives directly:
|
|
||||||
bridges, TAP devices, MAC addresses, static IPs, and Linux processes.
|
bridges, TAP devices, MAC addresses, static IPs, and Linux processes.
|
||||||
|
|
||||||
> Your LAN is the fabric.
|
> Your LAN is the fabric.
|
||||||
|
|
@ -24,10 +23,10 @@ bridges, TAP devices, MAC addresses, static IPs, and Linux processes.
|
||||||
|
|
||||||
## ☁️ What MNMIVM-SE Is
|
## ☁️ What MNMIVM-SE Is
|
||||||
|
|
||||||
* A **local VM cloud** built directly on your LAN
|
- A **local VM cloud** built directly on your LAN
|
||||||
* A **process-native control plane**
|
- A **process-native control plane**
|
||||||
* A **CLI-first infrastructure tool**
|
- A **CLI-first infrastructure tool**
|
||||||
* A Proxmox-style networking model **without Proxmox**
|
- A Proxmox-style networking model **without Proxmox**
|
||||||
|
|
||||||
Each VM:
|
Each VM:
|
||||||
- Has a persistent MAC address
|
- Has a persistent MAC address
|
||||||
|
|
@ -41,19 +40,17 @@ Routers, firewalls, and switches see MNMIVM-SE VMs as **real machines**, not NAT
|
||||||
|
|
||||||
## 🧠 Control Plane Model
|
## 🧠 Control Plane Model
|
||||||
|
|
||||||
MNMIVM-SE **does have a control plane** — it’s just intentionally **minimal, local, and explicit**.
|
MNMIVM-SE **does have a control plane**, but it is intentionally **minimal, local, and explicit**.
|
||||||
|
|
||||||
The control plane is implemented as:
|
- Single CLI binary
|
||||||
- A single CLI binary
|
- File-backed state
|
||||||
- A file-backed state store
|
|
||||||
- Linux process lifecycle tracking
|
- Linux process lifecycle tracking
|
||||||
|
|
||||||
There is:
|
There is:
|
||||||
- No always-on daemon
|
- No daemon
|
||||||
- No API server
|
- No API server
|
||||||
- No database
|
- No database
|
||||||
- No reconciliation loop
|
- No reconciliation loop
|
||||||
- No scheduler service
|
|
||||||
|
|
||||||
Instead:
|
Instead:
|
||||||
|
|
||||||
|
|
@ -66,73 +63,128 @@ Instead:
|
||||||
> `/proc` is the source of truth.
|
> `/proc` is the source of truth.
|
||||||
> Each CLI command is a deliberate control action.
|
> Each CLI command is a deliberate control action.
|
||||||
|
|
||||||
This makes MNMIVM-SE closer to **early private IaaS** and **bare-metal virtualization** than modern hyperscaler platforms.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧱 Supported Host Operating Systems
|
## 🧱 Supported Host Operating Systems
|
||||||
|
|
||||||
MNMIVM-SE is conservative about host support and only documents what is tested.
|
|
||||||
|
|
||||||
### ✅ Supported
|
### ✅ Supported
|
||||||
|
|
||||||
| Host OS | Version |
|
| Host OS | Version |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| **Debian** | 12+ |
|
| Debian | 12+ |
|
||||||
| **Alpine Linux** | 3.22+ |
|
| Alpine Linux | 3.22+ |
|
||||||
|
|
||||||
### 🕒 Coming Soon
|
### 🕒 Coming Soon
|
||||||
|
|
||||||
| Host OS | Notes |
|
| Host OS | Notes |
|
||||||
|------|------|
|
|------|------|
|
||||||
| Ubuntu | Netplan-based host networking support planned |
|
| Ubuntu | Netplan-based host networking planned |
|
||||||
|
|
||||||
> Ubuntu is not currently documented due to netplan-specific bridge handling.
|
Ubuntu is undocumented for now due to netplan differences, but it should not be much different in setting up a bridged network interface.
|
||||||
> Support will be added, but is not a top priority.
|
|
||||||
|
|
||||||
### ❌ Not Supported
|
### ❌ Not Supported
|
||||||
|
|
||||||
- Wi-Fi–only hosts
|
- Wi-Fi–only hosts
|
||||||
- WSL / nested hypervisors
|
- WSL / nested hypervisors
|
||||||
- Desktop laptop setups expecting NAT
|
- Desktop NAT-based setups
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 Debian Host Requirements (12+)
|
||||||
|
|
||||||
|
Install required packages:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install -y \
|
||||||
|
qemu-system-x86 \
|
||||||
|
golang \
|
||||||
|
qemu-utils \
|
||||||
|
qemu-bridge-helper \
|
||||||
|
cloud-image-utils \
|
||||||
|
genisoimage \
|
||||||
|
bridge-utils \
|
||||||
|
iproute2 \
|
||||||
|
iptables \
|
||||||
|
curl \
|
||||||
|
ca-certificates \
|
||||||
|
git \
|
||||||
|
build-essential
|
||||||
|
````
|
||||||
|
|
||||||
|
Optional but recommended:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install -y cpu-checker
|
||||||
|
kvm-ok
|
||||||
|
```
|
||||||
|
|
||||||
|
Ensure KVM is available:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls -l /dev/kvm
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔌 TUN/TAP Kernel Module (Required)
|
||||||
|
|
||||||
|
MNMIVM-SE requires the **TUN/TAP** kernel module.
|
||||||
|
|
||||||
|
Verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls -l /dev/net/tun
|
||||||
|
```
|
||||||
|
|
||||||
|
If missing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo modprobe tun
|
||||||
|
```
|
||||||
|
|
||||||
|
Persist on boot:
|
||||||
|
|
||||||
|
### Debian
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo tun | sudo tee /etc/modules-load.d/tun.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
### Alpine
|
||||||
|
|
||||||
|
```bash
|
||||||
|
echo tun | sudo tee -a /etc/modules
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧱 Architecture Overview
|
## 🧱 Architecture Overview
|
||||||
|
|
||||||
* **QEMU + KVM**
|
|
||||||
* **Linux bridge (`br0`)**
|
|
||||||
* **TAP devices**
|
|
||||||
* **Cloud-init seed ISO**
|
|
||||||
* **Static IP networking**
|
|
||||||
* **VNC console for recovery**
|
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
/var/lib/microvm/
|
/var/lib/microvm/
|
||||||
├── images/
|
├── images/
|
||||||
└── vms/
|
└── vms/
|
||||||
└── vm1/
|
└── vm1/
|
||||||
├── disk.qcow2
|
├── disk.qcow2
|
||||||
├── seed.iso
|
├── seed.iso
|
||||||
├── pubkey.pub
|
├── pubkey.pub
|
||||||
├── os.name
|
├── os.name
|
||||||
├── vm.ip
|
├── vm.ip
|
||||||
├── vm.mac
|
├── vm.mac
|
||||||
├── vnc.port
|
├── vnc.port
|
||||||
└── vm.pid
|
└── vm.pid
|
||||||
|
```
|
||||||
|
|
||||||
````
|
No libvirt.
|
||||||
|
No XML.
|
||||||
No libvirt.
|
|
||||||
No XML.
|
|
||||||
No daemon.
|
No daemon.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🌐 Host Networking Requirements (CRITICAL)
|
## 🌐 Host Networking Requirements (CRITICAL)
|
||||||
|
|
||||||
MNMIVM-SE requires a **proper Linux bridge**.
|
MNMIVM-SE requires a **Linux bridge**.
|
||||||
|
|
||||||
### Example: `/etc/network/interfaces` (Debian)
|
### Example: `/etc/network/interfaces` (Debian)
|
||||||
|
|
||||||
|
|
@ -152,12 +204,12 @@ iface br0 inet static
|
||||||
bridge_ports ens18
|
bridge_ports ens18
|
||||||
bridge_stp off
|
bridge_stp off
|
||||||
bridge_fd 0
|
bridge_fd 0
|
||||||
````
|
```
|
||||||
|
|
||||||
**Rules that must be followed:**
|
Rules:
|
||||||
|
|
||||||
* The host IP must live on `br0`
|
* Host IP must live on `br0`
|
||||||
* The physical NIC must have no IP
|
* Physical NIC has no IP
|
||||||
* Wi-Fi cannot be bridged
|
* Wi-Fi cannot be bridged
|
||||||
* VMs attach via TAP devices
|
* VMs attach via TAP devices
|
||||||
|
|
||||||
|
|
@ -165,22 +217,14 @@ iface br0 inet static
|
||||||
|
|
||||||
## 🔥 Kernel Bridge Filtering (THIS WILL BREAK VMs)
|
## 🔥 Kernel Bridge Filtering (THIS WILL BREAK VMs)
|
||||||
|
|
||||||
Linux defaults can silently block bridged traffic.
|
Check:
|
||||||
|
|
||||||
This **must** be disabled:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cat /proc/sys/net/bridge/bridge-nf-call-iptables
|
cat /proc/sys/net/bridge/bridge-nf-call-iptables
|
||||||
# must be 0
|
# must be 0
|
||||||
```
|
```
|
||||||
|
|
||||||
If set to `1`, VMs will:
|
Fix (runtime):
|
||||||
|
|
||||||
* Boot successfully
|
|
||||||
* Have valid IPs
|
|
||||||
* Be completely unreachable
|
|
||||||
|
|
||||||
### Fix (runtime)
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo sysctl -w net.bridge.bridge-nf-call-iptables=0
|
sudo sysctl -w net.bridge.bridge-nf-call-iptables=0
|
||||||
|
|
@ -188,11 +232,10 @@ sudo sysctl -w net.bridge.bridge-nf-call-ip6tables=0
|
||||||
sudo sysctl -w net.bridge.bridge-nf-call-arptables=0
|
sudo sysctl -w net.bridge.bridge-nf-call-arptables=0
|
||||||
```
|
```
|
||||||
|
|
||||||
### Persistent fix
|
Persist:
|
||||||
|
|
||||||
`/etc/sysctl.d/99-bridge.conf`:
|
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
|
# /etc/sysctl.d/99-bridge.conf
|
||||||
net.bridge.bridge-nf-call-iptables = 0
|
net.bridge.bridge-nf-call-iptables = 0
|
||||||
net.bridge.bridge-nf-call-ip6tables = 0
|
net.bridge.bridge-nf-call-ip6tables = 0
|
||||||
net.bridge.bridge-nf-call-arptables = 0
|
net.bridge.bridge-nf-call-arptables = 0
|
||||||
|
|
@ -202,11 +245,8 @@ net.bridge.bridge-nf-call-arptables = 0
|
||||||
|
|
||||||
## 🔐 QEMU Bridge Permissions
|
## 🔐 QEMU Bridge Permissions
|
||||||
|
|
||||||
QEMU must be allowed to attach TAP devices.
|
|
||||||
|
|
||||||
### `/etc/qemu/bridge.conf`
|
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
|
# /etc/qemu/bridge.conf
|
||||||
allow br0
|
allow br0
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -220,10 +260,6 @@ ls -l /usr/lib/qemu/qemu-bridge-helper
|
||||||
|
|
||||||
## 🐧 Alpine Linux Host Notes (3.22+)
|
## 🐧 Alpine Linux Host Notes (3.22+)
|
||||||
|
|
||||||
Alpine does not ship a hypervisor stack by default.
|
|
||||||
|
|
||||||
Install required packages:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
apk add \
|
apk add \
|
||||||
qemu-system-x86_64 \
|
qemu-system-x86_64 \
|
||||||
|
|
@ -237,20 +273,14 @@ apk add \
|
||||||
Notes:
|
Notes:
|
||||||
|
|
||||||
* `cdrkit` provides `genisoimage`
|
* `cdrkit` provides `genisoimage`
|
||||||
* `bridge-utils` provides `brctl`
|
* No libvirt or background services
|
||||||
* `qemu-hw-display-virtio-vga` is required for VNC
|
|
||||||
* No libvirt or services are used
|
|
||||||
* OpenRC is sufficient
|
* OpenRC is sufficient
|
||||||
|
|
||||||
Alpine works well as a **minimal KVM host** once assembled.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚙️ Server Edition Configuration (Code-Level)
|
## ⚙️ Server Edition Configuration (Code-Level)
|
||||||
|
|
||||||
Networking and sizing are configured **in code**, not via runtime flags.
|
Edit constants in `main.go`:
|
||||||
|
|
||||||
Edit these constants in `main.go` (around lines 25–30):
|
|
||||||
|
|
||||||
```go
|
```go
|
||||||
// Networking
|
// Networking
|
||||||
|
|
@ -266,13 +296,11 @@ memMB = "1024"
|
||||||
cpus = "1"
|
cpus = "1"
|
||||||
```
|
```
|
||||||
|
|
||||||
This keeps runtime behavior explicit and predictable.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧰 CLI Usage
|
## 🧰 CLI Usage
|
||||||
|
|
||||||
### Create a VM
|
### Create VM
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mnmivm-se create vm1 \
|
sudo mnmivm-se create vm1 \
|
||||||
|
|
@ -281,25 +309,25 @@ sudo mnmivm-se create vm1 \
|
||||||
--ip 192.168.86.53
|
--ip 192.168.86.53
|
||||||
```
|
```
|
||||||
|
|
||||||
### Start a VM
|
### Start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mnmivm-se start vm1
|
sudo mnmivm-se start vm1
|
||||||
```
|
```
|
||||||
|
|
||||||
### SSH in
|
### SSH
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ssh debian@192.168.86.53
|
ssh debian@192.168.86.53
|
||||||
```
|
```
|
||||||
|
|
||||||
### Stop a VM
|
### Stop
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mnmivm-se stop vm1
|
sudo mnmivm-se stop vm1
|
||||||
```
|
```
|
||||||
|
|
||||||
### Update cloud-init (SSH key / IP)
|
### Update cloud-init
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mnmivm-se update-cloud vm1 \
|
sudo mnmivm-se update-cloud vm1 \
|
||||||
|
|
@ -312,33 +340,32 @@ sudo mnmivm-se update-cloud vm1 \
|
||||||
## 🔑 Security Model
|
## 🔑 Security Model
|
||||||
|
|
||||||
* SSH key–only access
|
* SSH key–only access
|
||||||
* No password authentication
|
* No passwords
|
||||||
* No root login
|
* No root login
|
||||||
* Static IPs (no DHCP ambiguity)
|
* Static IPs
|
||||||
* MAC addresses pinned via cloud-init
|
* Pinned MAC addresses
|
||||||
* VNC console for recovery only
|
* VNC console for recovery only
|
||||||
|
|
||||||
This follows **server-grade discipline**, not container ergonomics.
|
Security is **intentional by default**.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚠️ What MNMIVM-SE Is Not
|
## ⚠️ What MNMIVM-SE Is Not
|
||||||
|
|
||||||
* ❌ A managed cloud service
|
* A managed cloud
|
||||||
* ❌ A multi-tenant platform
|
* A multi-tenant platform
|
||||||
* ❌ A scheduler or orchestrator
|
* A scheduler
|
||||||
* ❌ A UI-driven system
|
* A UI-driven system
|
||||||
* ❌ A laptop-friendly NAT tool
|
* A NAT-based laptop tool
|
||||||
|
|
||||||
If you want **policy, HA, quotas, tenants**, use Proxmox or OpenStack.
|
If you want policy and HA, use Proxmox or OpenStack.
|
||||||
|
If you want **direct infrastructure control**, use MNMIVM-SE.
|
||||||
If you want **direct control over real infrastructure**, MNMIVM-SE is the tool.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🐧 Why MNMIVM-SE Exists
|
## 🐧 Why MNMIVM-SE Exists
|
||||||
|
|
||||||
Because sometimes you don’t want:
|
Because sometimes you don't want:
|
||||||
|
|
||||||
* libvirt
|
* libvirt
|
||||||
* XML
|
* XML
|
||||||
|
|
@ -350,16 +377,15 @@ You want:
|
||||||
|
|
||||||
> “Put a VM on my LAN, give it an IP, and let me build infrastructure.”
|
> “Put a VM on my LAN, give it an IP, and let me build infrastructure.”
|
||||||
|
|
||||||
MNMIVM-SE does exactly that — and nothing more.
|
MNMIVM-SE does exactly that.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### ⚠️ Final Note
|
### ⚠️ Final Note
|
||||||
|
|
||||||
If you break networking with MNMIVM-SE, it isn’t a bug.
|
If networking breaks, it isn't a bug.
|
||||||
|
|
||||||
It’s Linux doing exactly what you told it to do.
|
It's Linux doing exactly what you told it to do.
|
||||||
|
|
||||||
And that’s the point.
|
|
||||||
|
|
||||||
|
And that's the point.
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue