Proxmox VE/IBM-306m

Installing PVE 5.2 on ibm 306m

Install Debian 9.4 (Stretch) Release
use lvm

 disable X desktop
 disable print
 enable ssh server
 enable standard

Once install is complete; Ensure all of the required networking is setup. enp4s0 should be on the public (600) vlan, enp5s0 on the private (601) vlan. If you forgot to deselect X desktop during install, then disable gdm, this is best done by: systemctl set-default multi-user.target

Enable non-free package source:

 apt-get install software-properties-common
 add-apt-repository non-free

Update the packages

 apt-get update

Install a few management packages:

 apt-get install sudo snmpd snmp-mibs-downloader ipmitool freeipmi-tools lm-sensors

Add the kernel modules for IPMI

 modprobe ipmi_si type=kcs ports=0xCA2 regspacings=1
 modprobe ipmi_devintf
 modprobe ipmi_msghandler
 modprobe ipmi_poweroff
 modprobe ipmi_watchdog
 modprobe smsc47m1
 modprobe smsc47m192
 modprobe lm85

Configure IPMI if not already configured in the bios (BMC must be ENABLED in the bios)

 ipmitool -I open lan set 1 ipsrc static
 ipmitool -I open lan set 1 ipaddr 10.246.254.XXX
 ipmitool -I open lan set 1 netmask 255.255.0.0
 ipmitool -I open lan set 1 defgw ipaddr 10.246.0.1
 ipmitool -I open lan set 1 access on
 ipmitool -I open user set password 2 STRONG_SHARED_PASSWORD
 ipmitool lan print

Make it so that snmpd listens on all interfaces (edit /etc/snmp/snmpd.conf)

 ###############################################################################
 #
 # AGENT BEHAVIOUR
 # 
 
 # Listen for connections from the local system only
 #agentAddress udp:127.0.0.1:161
 # Listen for connections on all interfaces (both IPv4 *and* IPv6)
 agentAddress udp:161,udp6:[::1]:161

Install the librenms agent for Proxxmox VE host monitoring:

 wget https://raw.githubusercontent.com/librenms/librenms-agent/master/agent-local/proxmox -O /usr/local/bin/librenms-agent-proxmox
 chmod +x /usr/local/bin/librenms-agent-proxmox
 echo "extend proxmox /usr/local/bin/librenms-agent-proxmox" >> /etc/snmp/snmpd.conf
 service snmpd restart

We are now ready to install Proxmox VE follow the instructions here:
https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Stretch

remove the PVE subscription source

 rm /etc/apt/sources.list.d/pve-enterprise.list

For configuring Ceph the following guide should be used
https://pve.proxmox.com/wiki/Ceph_Server#Installation_of_Proxmox_VE

Ceph Loop Hacks

THIS IS PROBABLY DANGEROUS PROCEED WITH CAUTION

 mkdir /ceph-osd
 fallocate -l 40G /ceph-osd/vdisk1.img
 losetup /dev/loop101 /ceph-osd/vdisk101.img # Start loop devices at 100 to avoid interfering with VMs

 ceph-disk prepare --zap-disk --cluster ceph --cluster-uuid 37048cb8-34cd-4525-9da9-6db9a1c89f80 --bluestore --osd-id {osd-id to use} /dev/loop101
 ceph-disk activate /dev/loop101

It may be worthwhile to follow the systemd.link process to wire (and rename) int interfaces to that regardless of the machine type that we have constant expectaions. see https://www.freedesktop.org/software/systemd/man/systemd.link.html for more information