DomU Setup

Encrypted disk stuff

root@gadolinium:~# lvcreate -L 1G -n bronze-swap dom0.root
  Logical volume "bronze-swap" created
root@gadolinium:~# lvcreate -L 4G -n bronze-disk dom0.root
  Logical volume "bronze-disk" created
root@gadolinium:~# lvcreate -L 100M -n bronze-boot dom0.root
  Logical volume "bronze-boot" created

root@gadolinium:~# cryptsetup luksFormat /dev/dom0.root/bronze-disk 

WARNING!
========
This will overwrite data on /dev/dom0.root/bronze-disk irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: 
Verify passphrase: 

root@gadolinium:~# cryptsetup luksOpen /dev/dom0.root/bronze-disk luks-bronze-disk
Enter passphrase for /dev/dom0.root/bronze-disk: 

root@gadolinium:~# /lib/cryptsetup/scripts/decrypt_derived luks-bronze-disk | cryptsetup luksFormat --key-file=- /dev/dom0.root/bronze-swap

root@gadolinium:~# /lib/cryptsetup/scripts/decrypt_derived luks-bronze-disk | cryptsetup luksOpen --key-file=- /dev/dom0.root/bronze-swap luks-bronze-swap

root@gadolinium:~# dd if=/dev/zero of=/dev/mapper/luks-bronze-disk bs=4M
dd: writing `/dev/mapper/luks-bronze-disk': No space left on device
1024+0 records in
1023+0 records out
4293914624 bytes (4.3 GB) copied, 86.6478 s, 49.6 MB/s

root@gadolinium:~# dd if=/dev/zero of=/dev/mapper/luks-bronze-swap bs=4M
dd: writing `/dev/mapper/luks-bronze-swap': No space left on device
256+0 records in
255+0 records out
1072689152 bytes (1.1 GB) copied, 22.1611 s, 48.4 MB/s

root@gadolinium:~# xen-create-image --hostname=bronze.club.cc.cmu.edu --memory=768Mb --fs=ext4 --ip=128.237.157.16 --mac=00:00:80:ed:9d:10 --broadcast=128.237.157.255 --gateway=128.237.157.1 --netmask=255.255.255.0 --nohosts --image-dev=/dev/mapper/luks-bronze-disk --swap-dev=/dev/mapper/luks-bronze-swap 

WARNING
-------

  You appear to have a missing vif-script, or network-script, in the
 Xen configuration file /etc/xen/xend-config.sxp.

  Please fix this and restart Xend, or your guests will not be able
 to use any networking!


General Information
--------------------
Hostname       :  bronze.club.cc.cmu.edu
Distribution   :  wheezy
Mirror         :  http://ftp.club.cc.cmu.edu/pub/debian/
Root Device    :  /dev/mapper/luks-bronze-disk
Swap Device    :  /dev/mapper/luks-bronze-swap
Partitions     :  Image type     :  full
Memory size    :  768Mb
Kernel path    :  /boot/vmlinuz-2.6.32-5-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.32-5-xen-amd64

Networking Information
----------------------
IP Address 1   : 128.237.157.16 [MAC: 00:00:80:ed:9d:10]
Netmask        : 255.255.255.0
Broadcast      : 128.237.157.255
Gateway        : 128.237.157.1


Creating ext4 filesystem on /dev/mapper/luks-bronze-disk
Done

Creating swap on /dev/mapper/luks-bronze-swap
Done
Installation method: debootstrap
Done

Running hooks
Done

No role scripts were specified.  Skipping

Creating Xen configuration file
Done
Setting up root password
Generating a password for the new guest.
All done


Logfile produced at:
         /var/log/xen-tools/bronze.club.cc.cmu.edu.log

Installation Summary
---------------------
Hostname        :  bronze.club.cc.cmu.edu
Distribution    :  wheezy
IP-Address(es)  :  128.237.157.16 
RSA Fingerprint :  5b:3c:27:44:22:ec:61:4f:82:be:1e:f4:06:2a:c7:46
Root Password   :  xxxxxxxx

root@gadolinium:~# mv /etc/xen/bronze.club.cc.cmu.edu.cfg /etc/xen/bronze
root@gadolinium:~# sed -i 's/bronze\.club\.cc\.cmu\.edu/bronze/g' /etc/xen/bronze

root@gadolinium:~# mkfs.ext4 /dev/dom0.root/bronze-boot 
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
25688 inodes, 102400 blocks
5120 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
13 block groups
8192 blocks per group, 8192 fragments per group
1976 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729

Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Chroot preparation

root@gadolinium:~# mkdir /tmp/bronze
root@gadolinium:~# mount /dev/mapper/luks-bronze-disk /tmp/bronze
root@gadolinium:~# mount /dev/dom0.root/bronze-boot /tmp/bronze/boot

root@gadolinium:~# mount --bind /dev /tmp/bronze/dev
root@gadolinium:~# mount --bind /dev/pts /tmp/bronze/dev/pts
root@gadolinium:~# mount -t proc proc /tmp/bronze/proc
root@gadolinium:~# mount -t sysfs sysfs /tmp/bronze/sys

root@gadolinium:~# chroot /tmp/bronze

root@bronze:~# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Pygrub

root@gadolinium:/# dpkg-divert --rename --divert /sbin/start-stop-daemon.real --local --add /sbin/start-stop-daemon
Adding 'local diversion of /sbin/start-stop-daemon to /sbin/start-stop-daemon.real'
root@gadolinium:/# echo -e '#!/bin/sh\necho invoked fake start-stop-daemon' > /sbin/start-stop-daemon
root@gadolinium:/# chmod a+x /sbin/start-stop-daemon

root@gadolinium:/# apt-get -o Apt::Install-Recommends=false install linux-image-3.2.0-4-amd64 grub-legacy cryptsetup
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  cryptsetup-bin dmsetup gettext-base grub-common initramfs-tools klibc-utils
  libasprintf0c2 libcryptsetup4 libdevmapper1.02.1 libfreetype6 libfuse2
  libklibc libuuid-perl linux-base
Suggested packages:
  dosfstools multiboot-doc grub-emu xorriso desktop-base grub-legacy-doc
  multiboot mdadm bash-completion fuse linux-doc-3.2 debian-kernel-handbook
  grub-pc extlinux lilo
Recommended packages:
  kbd console-setup busybox busybox-static os-prober firmware-linux-free
The following NEW packages will be installed:
  cryptsetup cryptsetup-bin dmsetup gettext-base grub-common grub-legacy
  initramfs-tools klibc-utils libasprintf0c2 libcryptsetup4 libdevmapper1.02.1
  libfreetype6 libfuse2 libklibc libuuid-perl linux-base
  linux-image-3.2.0-4-amd64
0 upgraded, 17 newly installed, 0 to remove and 4 not upgraded.
Need to get 27.7 MB of archives.
After this operation, 120 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libasprintf0c2 amd64 0.18.1.1-9 [26.8 kB]
Get:2 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main dmsetup amd64 2:1.02.74-8 [67.9 kB]
Get:3 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libdevmapper1.02.1 amd64 2:1.02.74-8 [130 kB]
Get:4 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libfreetype6 amd64 2.4.9-1.1 [451 kB]
Get:5 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libfuse2 amd64 2.9.0-2+deb7u1 [144 kB]
Get:6 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libuuid-perl amd64 0.02-5 [9742 B]
Get:7 http://security.debian.org/ wheezy/updates/main linux-image-3.2.0-4-amd64 amd64 3.2.65-1+deb7u1 [23.5 MB]
Get:8 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main linux-base all 3.5 [34.3 kB]
Get:9 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libklibc amd64 2.0.1-3.1 [56.9 kB]
Get:10 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main klibc-utils amd64 2.0.1-3.1 [192 kB]
Get:11 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main initramfs-tools all 0.109.1 [91.3 kB]
Get:12 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main gettext-base amd64 0.18.1.1-9 [154 kB]
Get:13 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main libcryptsetup4 amd64 2:1.4.3-4 [94.0 kB]
Get:14 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main cryptsetup-bin amd64 2:1.4.3-4 [153 kB]
Get:15 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main cryptsetup amd64 2:1.4.3-4 [128 kB]
Get:16 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main grub-common amd64 1.99-27+deb7u2 [1533 kB]
Get:17 http://ftp.club.cc.cmu.edu/pub/debian/ wheezy/main grub-legacy amd64 0.97-67 [924 kB]
Fetched 27.7 MB in 6s (4548 kB/s)                                              
Preconfiguring packages ...
Selecting previously unselected package libasprintf0c2:amd64.
(Reading database ... 13046 files and directories currently installed.)
Unpacking libasprintf0c2:amd64 (from .../libasprintf0c2_0.18.1.1-9_amd64.deb) ...
Selecting previously unselected package dmsetup.
Unpacking dmsetup (from .../dmsetup_2%3a1.02.74-8_amd64.deb) ...
Selecting previously unselected package libdevmapper1.02.1:amd64.
Unpacking libdevmapper1.02.1:amd64 (from .../libdevmapper1.02.1_2%3a1.02.74-8_amd64.deb) ...
Selecting previously unselected package libfreetype6:amd64.
Unpacking libfreetype6:amd64 (from .../libfreetype6_2.4.9-1.1_amd64.deb) ...
Selecting previously unselected package libfuse2:amd64.
Unpacking libfuse2:amd64 (from .../libfuse2_2.9.0-2+deb7u1_amd64.deb) ...
Selecting previously unselected package libuuid-perl.
Unpacking libuuid-perl (from .../libuuid-perl_0.02-5_amd64.deb) ...
Selecting previously unselected package linux-base.
Unpacking linux-base (from .../linux-base_3.5_all.deb) ...
Selecting previously unselected package libklibc.
Unpacking libklibc (from .../libklibc_2.0.1-3.1_amd64.deb) ...
Selecting previously unselected package klibc-utils.
Unpacking klibc-utils (from .../klibc-utils_2.0.1-3.1_amd64.deb) ...
Selecting previously unselected package initramfs-tools.
Unpacking initramfs-tools (from .../initramfs-tools_0.109.1_all.deb) ...
Selecting previously unselected package linux-image-3.2.0-4-amd64.
Unpacking linux-image-3.2.0-4-amd64 (from .../linux-image-3.2.0-4-amd64_3.2.65-1+deb7u1_amd64.deb) ...
Selecting previously unselected package gettext-base.
Unpacking gettext-base (from .../gettext-base_0.18.1.1-9_amd64.deb) ...
Selecting previously unselected package libcryptsetup4.
Unpacking libcryptsetup4 (from .../libcryptsetup4_2%3a1.4.3-4_amd64.deb) ...
Selecting previously unselected package cryptsetup-bin.
Unpacking cryptsetup-bin (from .../cryptsetup-bin_2%3a1.4.3-4_amd64.deb) ...
Selecting previously unselected package cryptsetup.
Unpacking cryptsetup (from .../cryptsetup_2%3a1.4.3-4_amd64.deb) ...
Selecting previously unselected package grub-common.
Unpacking grub-common (from .../grub-common_1.99-27+deb7u2_amd64.deb) ...
Selecting previously unselected package grub-legacy.
Unpacking grub-legacy (from .../grub-legacy_0.97-67_amd64.deb) ...
Processing triggers for man-db ...
Setting up libasprintf0c2:amd64 (0.18.1.1-9) ...
Setting up libfreetype6:amd64 (2.4.9-1.1) ...
Setting up libfuse2:amd64 (2.9.0-2+deb7u1) ...
Setting up libuuid-perl (0.02-5) ...
Setting up linux-base (3.5) ...
Setting up libklibc (2.0.1-3.1) ...
Setting up klibc-utils (2.0.1-3.1) ...
Setting up initramfs-tools (0.109.1) ...
update-initramfs: deferring update (trigger activated)
Setting up linux-image-3.2.0-4-amd64 (3.2.65-1+deb7u1) ...
Running depmod.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.2.0-4-amd64 /boot/vmlinuz-3.2.0-4-amd64
update-initramfs: Generating /boot/initrd.img-3.2.0-4-amd64
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: failed to detect canonical device of /dev/xvda2
cryptsetup: WARNING: could not determine root device from /etc/fstab
loadkeys is missing. Please install the 'kbd' package.
Setting up gettext-base (0.18.1.1-9) ...
Setting up libdevmapper1.02.1:amd64 (2:1.02.74-8) ...
Setting up libcryptsetup4 (2:1.4.3-4) ...
Setting up cryptsetup-bin (2:1.4.3-4) ...
Setting up grub-common (1.99-27+deb7u2) ...
Setting up grub-legacy (0.97-67) ...
Setting up dmsetup (2:1.02.74-8) ...
update-initramfs: deferring update (trigger activated)
Setting up cryptsetup (2:1.4.3-4) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.2.0-4-amd64
df: Warning: cannot read table of mounted file systems: No such file or directory
cryptsetup: WARNING: failed to detect canonical device of /dev/xvda2
cryptsetup: WARNING: could not determine root device from /etc/fstab
loadkeys is missing. Please install the 'kbd' package.

root@gadolinium:/# mkdir /boot/grub
root@gadolinium:/# update-grub
Searching for GRUB installation directory ... found: /boot/grub
Probing devices to guess BIOS drives. This may take a long time.
Searching for default file ... Generating /boot/grub/default file and setting the default boot entry to 0
Searching for GRUB installation directory ... found: /boot/grub
Testing for an existing GRUB menu.lst file ... 


Generating /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Found kernel: /vmlinuz-3.2.0-4-amd64
Updating /boot/grub/menu.lst ... done

root@gadolinium:/# vi /boot/grub/menu.lst
# NORMAL:
#   Fix "# kopt=" line; -> # kopt=root=/dev/xvda2 ro console=hvc0
#   Fix "# groot=" line; -> # groot=(hd0,1)
# ENCRYPTED:
#   Fix "# kopt=" line; -> # kopt=root=/dev/mapper/rootdev cryptopts=source=/dev/xvda2,target=rootdev ro console=hvc0
#   Fix "# groot=" line; -> # groot=(hd0,2)

root@gadolinium:/# update-grub

root@gadolinium:/# rm /sbin/start-stop-daemon
root@gadolinium:/# dpkg-divert --rename --remove /sbin/start-stop-daemon
Removing 'local diversion of /sbin/start-stop-daemon to /sbin/start-stop-daemon.real'

More encrypted disk stuff

root@gadolinium:/# sed -i -e 's/xvda1/mapper\/swapdev/' -e 's/xvda2/mapper\/rootdev/' /etc/fstab
root@gadolinium:/# echo /dev/xvda3 /boot ext4 defaults 0 2 >> /etc/fstab

root@gadolinium:/# echo rootdev /dev/xvda2 none luks,tries=0 >> /etc/crypttab
root@gadolinium:/# echo swapdev /dev/xvda1 rootdev luks,keyscript=/lib/cryptsetup/scripts/decrypt_derived >> /etc/crypttab

Chroot teardown

root@gadolinium:/# exit
exit

root@gadolinium:~# umount /tmp/bronze/sys
root@gadolinium:~# umount /tmp/bronze/proc
root@gadolinium:~# umount /tmp/bronze/dev/pts
root@gadolinium:~# umount /tmp/bronze/dev    
root@gadolinium:~# umount /tmp/bronze/boot
root@gadolinium:~# umount /tmp/bronze     
root@gadolinium:~# rmdir /tmp/bronze

More pygrub

root@gadolinium:~# vim /etc/xen/bronze
# Comment out "kernel = ..." and "ramdisk = ..." lines
# Add "bootloader = '/usr/lib/xen-4.0/bin/pygrub'"

Even more encrypted disk stuff

root@gadolinium:~# vim /etc/xen/bronze
# Fix up "disk = ..." list to 1) add boot-device as first entry, and 2) use the LVM devices instead of the opened crypt block devices
# Comment out the "root = ..." line

root@gadolinium:~# cryptsetup luksClose luks-bronze-swap
root@gadolinium:~# cryptsetup luksClose luks-bronze-disk

root@gadolinium:~# xm create -c bronze

Normal domU setup

See Common Maintenance Tasks/Building Xen Domains -> Wheezy -> DomU. Start at step 5.2.

KDC setup

Installation

root@bronze:~# aptitude update
Hit http://debian.club.cc.cmu.edu wheezy-cclub Release.gpg
Hit http://debian.club.cc.cmu.edu wheezy-cclub Release       
Get: 1 http://mirrors.mit.edu wheezy Release.gpg [1655 B]    
Get: 2 http://mirrors.mit.edu wheezy-updates Release.gpg [836 B]
Get: 3 http://mirrors.mit.edu wheezy Release [168 kB]         
Hit http://security.debian.org wheezy/updates Release.gpg                   
Get: 4 http://debian.club.cc.cmu.edu wheezy-cclub/contrib Sources [7298 B]
Hit http://security.debian.org wheezy/updates Release                           
Hit http://debian.club.cc.cmu.edu wheezy-cclub/contrib amd64 Packages        
Hit http://security.debian.org wheezy/updates/main Sources                     
Get: 5 http://mirrors.mit.edu wheezy-updates Release [124 kB]                   
Hit http://security.debian.org wheezy/updates/contrib Sources
Hit http://security.debian.org wheezy/updates/non-free Sources                
Hit http://security.debian.org wheezy/updates/main amd64 Packages             
Hit http://security.debian.org wheezy/updates/contrib amd64 Packages          
Ign http://debian.club.cc.cmu.edu wheezy-cclub/contrib Translation-en
Hit http://security.debian.org wheezy/updates/non-free amd64 Packages
Hit http://security.debian.org wheezy/updates/contrib Translation-en
Get: 6 http://mirrors.mit.edu wheezy/main Sources [5971 kB]
Hit http://security.debian.org wheezy/updates/main Translation-en
Hit http://security.debian.org wheezy/updates/non-free Translation-en
Get: 7 http://mirrors.mit.edu wheezy/contrib Sources [47.7 kB]
Get: 8 http://mirrors.mit.edu wheezy/non-free Sources [93.5 kB]
Get: 9 http://mirrors.mit.edu wheezy/main amd64 Packages [5841 kB]
Get: 10 http://mirrors.mit.edu wheezy/contrib amd64 Packages [42.0 kB]
Get: 11 http://mirrors.mit.edu wheezy/non-free amd64 Packages [80.8 kB]
Get: 12 http://mirrors.mit.edu wheezy/contrib Translation-en [34.8 kB]
Get: 13 http://mirrors.mit.edu wheezy/main Translation-en [3848 kB]             
Get: 14 http://mirrors.mit.edu wheezy/non-free Translation-en [66.1 kB]         
Get: 15 http://mirrors.mit.edu wheezy-updates/main Sources [2929 B]             
Get: 16 http://mirrors.mit.edu wheezy-updates/contrib Sources [14 B]            
Get: 17 http://mirrors.mit.edu wheezy-updates/non-free Sources [14 B]           
Get: 18 http://mirrors.mit.edu wheezy-updates/main amd64 Packages [4126 B]      
Get: 19 http://mirrors.mit.edu wheezy-updates/contrib amd64 Packages [14 B]     
Get: 20 http://mirrors.mit.edu wheezy-updates/non-free amd64 Packages [14 B]    
Get: 21 http://mirrors.mit.edu wheezy-updates/contrib Translation-en [14 B]     
Get: 22 http://mirrors.mit.edu wheezy-updates/main Translation-en [2942 B]      
Get: 23 http://mirrors.mit.edu wheezy-updates/non-free Translation-en [14 B]    
Fetched 16.3 MB in 18s (895 kB/s)                                               
                            
Current status: 1 update [+1].

# Just hit enter for both of the debconf questions.
# We will be blowing away the new database and master key the package creates.

root@bronze:~# aptitude install heimdal-kdc
The following NEW packages will be installed:
  heimdal-kdc libcap-ng0{a} libkdc2-heimdal{a} 
0 packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/229 kB of archives. After unpacking 571 kB will be used.
Do you want to continue? [Y/n/?] y
Preconfiguring packages ...
Heimdal KDC
-----------

Please enter the name of the local Kerberos realm.

Using the uppercase domain name is common. For instance, if the host name is 
host.example.org, then the realm will become EXAMPLE.ORG. The default for this 
host is CLUB.CC.CMU.EDU.

Local realm name: CLUB.CC.CMU.EDU


Heimdal can encrypt the key distribution center (KDC) data with a password. A 
hashed representation of this password will be stored in 
/var/lib/heimdal-kdc/m-key.

KDC password: 


Selecting previously unselected package libkdc2-heimdal:amd64.
(Reading database ... 30102 files and directories currently installed.)
Unpacking libkdc2-heimdal:amd64 (from .../libkdc2-heimdal_1.6~git20120403+dfsg1-2_amd64.deb) ...
Selecting previously unselected package libcap-ng0.
Unpacking libcap-ng0 (from .../libcap-ng0_0.6.6-2_amd64.deb) ...
Selecting previously unselected package heimdal-kdc.
Unpacking heimdal-kdc (from .../heimdal-kdc_1.6~git20120403+dfsg1-2_amd64.deb) ...
Processing triggers for man-db ...
Setting up libkdc2-heimdal:amd64 (1.6~git20120403+dfsg1-2) ...
Setting up libcap-ng0 (0.6.6-2) ...
Setting up heimdal-kdc (1.6~git20120403+dfsg1-2) ...
kstash: writing key to `/var/lib/heimdal-kdc/m-key'
Processing service `#krb_prop' ... not enabled (entry is commented out by user)
Starting Heimdal KDC: heimdal-kdc.
Starting Heimdal password server: kpasswdd.

# Start with disabled kadmind and kpasswdd.
# Until we've transitioned off of our existing KDCs, we're making slaves.

root@bronze:~# /etc/init.d/heimdal-kdc stop
Stopping Heimdal password server: kpasswdd.
Stopping Heimdal KDC: heimdal-kdc.

root@bronze:~# update-inetd --comment-chars \# --disable kerberos-adm
root@bronze:~# sed -i -e 's/^\(KPASSWDD_ENABLED=\).*/\1no/' /etc/default/heimdal-kdc 

root@bronze:~# kinit -S kadmin/admin kbare/admin
kbare/admin@CLUB.CC.CMU.EDU's Password: 

root@bronze:~# ktutil -k /etc/kdc.keytab get hprop/bronze.club.cc.cmu.edu
# XXX: couldn't netreg as would be required to use the correct name
root@bronze:~# ktutil -k /etc/kdc.keytab get hprop/calcium.club.cc.cmu.edu
root@bronze:~# kadmin ext -k /etc/kdc.keytab kadmin/admin

root@bronze:~# sed -i -e '/^#\?\(kerberos-adm\|krb_prop\)/{' -e 's/kdc\.keytab/&/' -e 't' -e 's/$/ --keytab=\/etc\/kdc.keytab/' -e '}' /etc/inetd.conf

Setup hprop of database from existing master KDC

Note that this is not needed for the long-term, but rather to support testing new KDCs while the master (or all production KDCs) are still running old software.

# Make sure the kdc is stopped.

root@bronze:~# kinit kbare/root
kbare/root@CLUB.CC.CMU.EDU's Password:

# Default path changed (in fact, Debian configuration even seems to put the master
# key in the wrong place).
# Err... actually it seems that various daemons don't agree on where it should be.

root@labrador:~# rm /var/lib/heimdal-kdc/log

root@bronze:~# scp -P 222 sodium:/var/heimdal/m-key /var/lib/heimdal-kdc/heimdal.mkey
m-key                                         100%   68     0.1KB/s   00:00    
root@bronze:~# ln -sf heimdal.mkey /var/lib/heimdal-kdc/m-key

root@labrador:~# echo "hpropd: ALL" >> /etc/hosts.deny
root@labrador:~# echo "hpropd: sodium.club.cc.cmu.edu sodium-old.club.cc.cmu.edu" >> /etc/hosts.allow

root@bronze:~# update-inetd --comment-chars \# --enable krb_prop

# XXX: would be bronze if DNS were set up correctly
root@bronze:~# ssh -p 222 sodium hprop calcium

Periodic hprop database updates

# XXX: until we've fully switched over to new KDCs, we should run the hprop command
# every 15 minutes or so from root's crontab.

sodium:~# EDITOR=vim crontab -e -u root
# Add to the end:  3,18,33,48 *    * * *   /usr/sbin/hprop labrador bloodhound
# Replacing "labrador" and "bloodhound" with whatever the correct hostnames are.

Configuration file updates

root@bronze:~# scp -P 222 sodium:/etc/heimdal-kdc/kadmind.acl /etc/heimdal-kdc/kadmind.acl 
kadmind.acl                                   100%  531     0.5KB/s   00:00    

root@bronze:~# vim /etc/heimdal-kdc/kdc.conf
# [kdc]
# require-preauth --> true
# check-ticket-addresses --> false
#
# [kadmin]
# require-preauth --> true

# XXX: do we need to listen on more ports?  Existing KDCs are listening
# on the krb4 port and the 524 port.  We've turned those off, but maybe
# we still want to listen so stuff attempting krb4 things fail quickly
# instead of taking a long timeout.

root@bronze:# /etc/init.d/heimdal-kdc start
Starting Heimdal KDC: heimdal-kdc.

root@bloodhound:~# aptitude install heimdal-docs
The following NEW packages will be installed:
  heimdal-docs 
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 98.4 kB of archives. After unpacking 115 kB will be used.
Get: 1 http://mirrors.mit.edu/debian/ wheezy/main heimdal-docs all 1.6~git20120403+dfsg1-2 [98.4 kB]
Fetched 98.4 kB in 0s (919 kB/s)  
Selecting previously unselected package heimdal-docs.
(Reading database ... 27498 files and directories currently installed.)
Unpacking heimdal-docs (from .../heimdal-docs_1.6~git20120403+dfsg1-2_all.deb) ...
Processing triggers for install-info ...
Processing triggers for man-db ...
Setting up heimdal-docs (1.6~git20120403+dfsg1-2) ...

root@bloodhound:~# aptitude clean

Testing

Add the following in the CLUB.CC.CMU.EDU = { ... } part of the [realms] section:

        # The following were added for testing wheezy KDCs.
        # Remove when done testing.
        master_kdc = sodium.club.cc.cmu.edu
        kdc = labrador.club.cc.cmu.edu
        kdc = bloodhound.club.cc.cmu.edu

This has been done to:

Remember to remove those lines from all hosts after testing is done!!!

And to update DNS (SRV records as well as standard fallback CNAMEs).

Setting up Kadmind/Kpasswdd

You need to extract the kadmin/admin keytab for Kadmind to work. For purposes of failing over, you should do this on all hosts.

kadmin -l ext -k /etc/kdc.keytab kadmin/admin

Then enable the service. ONLY DO THIS ON THE MASTER.

root@bloodhound:~# update-inetd --comment-chars \# --enable kerberos-adm

ONLY DO THIS ON THE MASTER. Enable Kpasswdd in /etc/default/heimdal-kdc. It should look like:

KPASSWDD_ENABLED=yes
KPASSWDD_PARAMS=""

Setting up Iprop master

/!\ Iprop turned out to be epicly broken. IIRC, the slave would just abort() when receiving certain ops from the master. Just use hprop unless it gets fixed.

On the master, create an empty log.

root@labrador:~# iprop-log truncate

The result should look like this (substituting the current date):

root@labrador:~# iprop-log last-version
version: 1
root@labrador:~# iprop-log dump
nop: ver = 1, timestamp = 2015-02-28 18:43:40, len = 0

Create a principal for the master and each of the slaves:

root@labrador:~# ktutil -k /etc/kdc.keytab get iprop/labrador.club.cc.cmu.edu
root@labrador:~# ktutil -k /etc/kdc.keytab get iprop/bloodhound.club.cc.cmu.edu

Put the principal names for the slaves in the slaves file:

root@labrador:~# cat > /var/lib/heimdal-kdc/slaves << EOF
iprop/bloodhound.club.cc.cmu.edu@CLUB.CC.CMU.EDU
EOF

Enable it (and set its commandline properly) in /etc/default/heimdal-kdc. It should look like:

MASTER_ENABLED=yes
MASTER_PARAMS="--slave-stats-file=/var/lib/heimdal-kdc/slave-stats"

Then start it:

root@bloodhound/etc/init.d/heimdal-kdc restart

Setting up Iprop slave

/!\ Iprop turned out to be epicly broken. IIRC, the slave would just abort() when receiving certain ops from the master. Just use hprop unless it gets fixed.

Extract keytabs for the master and each of the slaves.

root@bloodhound:~# kadmin ext -k /etc/kdc.keytab iprop/labrador.club.cc.cmu.edu
root@bloodhound:~# kadmin ext -k /etc/kdc.keytab iprop/bloodhound.club.cc.cmu.edu

Enable it (and set its commandline properly) in /etc/default/heimdal-kdc. It should look like:

SLAVE_ENABLED=yes
SLAVE_PARAMS="-k /etc/kdc.keytab labrador.club.cc.cmu.edu"

Note that the host name is the name of the master.

root@bloodhound:~# /etc/init.d/heimdal-kdc restart

kbare at CLUB.CC.CMU.EDU/Kerb Notes (last edited 2016-09-17 18:22:26 by kbare@CLUB.CC.CMU.EDU)