• Common Maintenance Tasks
• Building Xen Domains

Building a DomU

Netreg machine name with cl0x and an empty MAC to get an IP. Then use the IP to update that record with the correct MAC.

• note IP address => MAC function

  • just encode the IP in hex as the MAC address
  • ok, there will never be Cray ethernet hardware in B6

Implicit step:

• apt-get update
• apt-get dist-upgrade
  • If krb5-config gets upgraded, rsync /etc/krb5.conf from bromine
  • If kernel upgrade is required, then it takes a while

Create LVM volumes for the DomU

• pick a short name for the domU (eg atomic symbol)
• figure out short name of dom0 (hint: vgdisplay)
• lvcreate -L 1G -n [domU-short].root [dom0-short].root
• lvcreate -L 1G -n [domU-short].swap [dom0-short].root
• lvcreate -L 1G -n [domU-short].afscache [dom0-short].root

Mkasick's magic image, uncompress it onto the root device.

• on osmium (and most dom0s too)
• etch-i386-2007041000.img.bz2

• bzcat etch-etc.img.bz2 > /dev/[dom0-short].root/[domU-short].root

  • note: jfs filesystem

Make the swap partition.

• mkswap /dev/[dom0-short].root/[domU-short].swap

Make the afs cache filesystem.

• mkfs.ext3 /dev/[dom0-short].root/]domU-short].afscache

Mount the root filesystem (probably want to chroot).

• mkdir /mnt/[domU-short].root
• mount /dev/[dom0-short].root/[domU-short].root /mnt/[domU-short].root
• mount -o remount,resize /mnt/domU-short].root

  • if you made a > 1GB root LV

• chroot /mnt/[domU-short].root

Update config files on the new domU.

• Need to make sure they have the right IP, MAC, hostname, etc.
• /etc/hostname
• /etc/hosts
• /etc/network/interfaces
• /etc/fstab

Do magic on the domU to fix the change from Xen emulating device tty1 to hvc0 (could put in magic image eventually)

• /etc/inittab
  • change the relevant line (eg 1:2345:respawn:/sbin/getty 38400 tty1) to 1:2345:respawn:/sbin/getty 38400 hvc0
• /etc/securetty
  • add hvc0

Remove udev persistent net rules the domU.

• rm /etc/udev/rules.d/z25_persistent-net.rules

Xen configuration file on the dom0.

• /etc/xen/...
• symlink from auto, so starts on boot

If installing without AFS, don't have the AFS Cache in /etc/fstab. If afscache is ext3 instead of ext2, make that change in fstab as well.

Start the domain.

Upgrade packages.

• apt-get update
• apt-get dist-upgrade
• apt-get install linux-modules-2.6.24-1-xen-686
• apt-get install openafs-modules-2.6.24-1-xen-686
• apt-get install openafs-client
• apt-get install libpam-afs-session
  • max cache size for 1G afscache is 980400 kb
  • do not dynamically generate /afs
• get /afs/club/service/etc/skel/packages.etch.domU into /root
  • replaces old version in magic image
  • afs may or may not be working until reboot, may have to scp off another machine
• cat /root/packages.etch.domU | xargs apt-get install -y --force-yes
  • exim4 configuration
    • do not split config files
    • smarthost, no local mail
    • accept defaults until...

    • "visible domain name" => "club.cc.cmu.edu" (no machine name)

    • "outgoing smarthost" => "smtp.club.cc.cmu.edu" (add smtp)

    • accept the defaults for the rest
• you will probably want to reboot at this point

Create kerb instances for the machine.

• kinit you/admin
• kadmin add -r host/[domU].club.cc.cmu.edu
  • accept defaults
• kadmin ext host/[domU].club.cc.cmu.edu

If not using afs, copy the rsync script from a machine that does have afs and run it as kerb admin.

• kinit you/admin
• /afs/club.cc.cmu.edu/system/scripts/sh/newrsync-etch.sh

Run the new machine script as kerb admin.

• kinit you/admin
• /afs/club/system/scripts/sh/newmachine-etch.sh

By default only passwd.admin is allowed to log in.

• touch /etc/passwd.user
• will allow all users on the next /etc/passwd sync

Add to DNS

• /afs/club/service/dns/DB.club.cc.cmu.edu

Building a Dom0

Install etch:

Before "Partition disks":

#the actual cylinders don't really matter
#just make sure that hda1 starts at 1 and is 128M
fdisk /dev/hda
    /dev/hda1:   1-  260 83  (128 MB) # /boot
    /dev/hda2: 261-      8e           # lvm

modprobe dm-mod
pvcreate /dev/hda2
vgcreate xx.root /dev/hda2
lvcreate -L 1G -n xx.root xx.root      # /
lvcreate -L 512M -n xx.swap xx.root    # swap
#Size of xx.xensave should be the same as the amount of physical ram
lvcreate -L 512M -n xx.xensave xx.root # /var/lib/xen/save

During "Partition disks":
/boot should be jfs
xx.root should be jfs
xx.swap should be swap
xx.xensave should be jfs

Install linux-image-2.6-686.
During package selection, do not install base system.
Install grub to MBR.

After reboot:

dpkg-reconfigure debconf
Change priority to medium.

apt-get update
apt-get dist-upgrade
apt-get install vim

vi /boot/grub/menu.lst
#Replace \n with a new line
Add "serial --unit=0 --speed=9600\nterminal serial" before
"BEGIN AUTOMATIC KERNELS LIST".
Change "# kopt=root=/dev/mapper/xx.root-xx.root ro console=ttyS0".
Change "# xenhopt=com1=9600,8n1".
Change "# xenkopt=".

vi /etc/inittab
Uncomment "#TO:23:respawn:/sbin/getty -L ttyS0 9600 vt100".

#If on an IA32 system:
apt-get install bridge-utils libc6-xen linux-image-2.6-xen-686 xen-hypervisor-3.0.3-1-i386-pae

#If on an AMD64 system:
#apt-get install bridge-utils linux-image-2.6-xen-amd64 xen-hypervisor-3.0.3-1-amd64

vi /etc/default/xendomains
Change 'XENDOMAINS_SAVE=""'. #do not forget quotes

vi /etc/network/interfaces
Change "allow-hotplug eth0" to "auto br0".
Change "iface eth0 inet static" to "iface br0 inet static".
Add "bridge_ports eth0" under "gateway 128.237.157.1".

vi /etc/apt/apt.conf
Replace with 'APT::Default-Release "etch-cclub";'

vi /etc/apt/sources.list
Add "deb http://debian.club.cc.cmu.edu/debian/ etch-cclub contrib".

apt-get update
apt-get dist-upgrade

apt-get install linux-image-2.6.18-4-xen-686 linux-image-2.6.18-5-xen-686 linux-image-2.6.18-6-xen-686 linux-image-2.6.24-1-xen-686

apt-get install openssh-client
scp unix.club.cc.cmu.edu:/afs/club/service/etc/skel/packages.etch.dom* .
apt-get install `cat packages.etch.domU`
apt-get install `cat packages.etch.dom0`

#adjust hostname as necessary, username as necessary
kinit mkasick/admin
kadmin add -r host/osmium.club.cc.cmu.edu
kadmin ext host/osmium.club.cc.cmu.edu

#if this is a non-AFS domain
scp unix.club.cc.cmu.edu:/afs/club/system/scripts/sh/newrsync-etch.sh .
./newrsync-etch.sh
#endif

/afs/club/system/scripts/sh/newmachine-etch.sh

shutdown -r now