NETOPS / HOMELAB

Homelab Notes

A practical memory bank for the infrastructure I run, test and learn from at home. The goal is not to document every possible platform—it is to keep the checks, patterns and recovery thinking I actually want close at hand.

Know what is running

Start with inventory before troubleshooting. A lab becomes hard to maintain when services exist only in memory.

systemctl --type=service --state=runningdocker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}'ss -lntup

Virtualization & containers

Keep the boundary clear: host, VM, container, network and persistent storage. Know which layer owns the problem.

  • Record host and guest IPs.
  • Know where persistent volumes live.
  • Keep compose/config files in versioned storage.
  • Do not confuse a container restart with a repair.

Storage first aid

Capacity problems often surface as application failures. Check the boring things early.

df -hTdu -xh /var | sort -h | taillsblk -ffindmnt

Backups that can come back

A backup is only useful if the restore path is understood. Separate important data from disposable system state.

  • Identify irreplaceable data.
  • Keep more than one copy.
  • Keep one copy away from the primary host.
  • Test restores, not just backup jobs.
  • Write down encryption keys and recovery dependencies safely.

Observe before changing

Capture state before restarting everything. The evidence often disappears with the restart.

systemctl --failedjournalctl -p warning..alert -buptime && free -hip addr && ip route

Network path

Work outward: local interface, gateway, routing, DNS, remote port, application.

ip route get 1.1.1.1ping -c 3 <gateway>resolvectl query example.comcurl -I --connect-timeout 5 https://example.com

Updates without surprises

Know what you are changing and have a recovery route before major upgrades.

  • Read release notes for important services.
  • Snapshot only when a snapshot is actually useful.
  • Back up application data separately.
  • Update one layer at a time when possible.
  • Verify services after reboot.

Recovery order

When several things are down, restore dependencies before applications.

  1. Power and host health
  2. Storage
  3. Network and DNS
  4. Authentication / shared dependencies
  5. Core services
  6. Convenience services
Lab rule: document the weird failure after you solve it. The most valuable homelab note is usually the one written five minutes after discovering why something broke.