We use NUT to communicate and get status information from our UPSes. We also try to run upsmon on the machines that are plugged into a UPS, so that they can cleanly shutdown in the event of an extended power outage.

Topology

This information was updated as of 2022-05-07.

halite

graphite

garnet

APC Smart-UPS 2200 RM,
in rack 4, upper position

APC Smart-UPS 2200 RM,
in rack 2, upper position

APC Smart-UPS 2200 RM XL,
in rack 0

Serial number

ES0453001065

Serial number

AS1231144183

Serial number

JS0653001390

NUT master

virt-hp-03

NUT master

virt-hp-04

NUT master

storage-04

NUT slaves

componium, storage-05, transit-1, virt-hp-01, virt-hp-05, virt-hp-dev

NUT slaves

dsmcp, virt-hp-02, virt-hp-06

NUT slaves

backups-02, opc-hp-dev, transit-2

Other equipment

rack4-baytech

Other equipment

console network, pm2e-1, rack2-baytech, switch-01, weather

Other equipment

rack0-baytech

 

quartz

beryl

APC Smart-UPS 1500 RM,
in rack 4, lower position

APC Smart-UPS 1500 RM,
in rack 2, lower position

Serial number

AS0807130224

Serial number

AS0508335688

NUT master

opc-hp-01

NUT master

opc-hp-02

NUT slaves

lawrencium, opc-hp-03, opc-hp-05

NUT slaves

jocelynh, opc-hp-04, opc-hp-06

Other equipment

 

Other equipment

rack 2 KVM switch

Hosts in italics have not had NUT fully configured.

Setup

The cclub-hardware-configuration package will install the NUT packages required to monitor UPSes. However, monitoring is not automatically configured—this needs to be done manually, as the exact circumstances will depend on the particular machine.

Hosts that are directly connected to the UPS via USB/serial will run the NUT server as well as the NUT monitor. Hosts that are not directly connected will only run the NUT monitor and learn of UPS status by connecting to the server on a remote host.

Set up the NUT Server (upsd)

  1. Add the nut user to the dialout group. This is needed so that the UPS driver process can open the serial port device and to communicate with the UPS.

    adduser nut dialout
  2. Define the UPS in /etc/nut/ups.conf. E.g., to define a UPS named dilithium:

    [dilithium]
            driver = apcsmart
            port = /dev/ttyUSB0
            desc = "Lower APC Smart-UPS 1337 in rack 9"
  3. Enable NUT in its main "environment" file, /etc/nut/nut.conf. Change the MODE variable to netserver.

  4. Enable and start upsd.

    systemctl enable nut-server.service # assuming the host is running systemd
    service nut-server start
  5. Verify NUT is successfully communicating with the UPS.
    upsc «upsname»@localhost

Set up the NUT Monitor (upsmon)

  1. Add a MONITOR line to /etc/nut/upsmon.conf. It can take one two forms depending on whether the UPS monitoring interfaces is directly connected to the machine. For a directly connected machine (i.e., one that is also running upsd), this would look like:

    MONITOR «upsname»@localhost 1 localmon «password» master

    For a machine contacting a remote upsd:

    MONITOR «upsname»@«remotehost» 1 remotemon «password» slave

    The passwords for the localmon and remotemon users are available in /etc/nut/upsd.users on the directly connected machine.

  2. Enable NUT in its main "environment" file, /etc/nut/nut.conf, if you did not already do so because the host is also running the NUT server. Change the MODE variable from none to netclient.

  3. Enable and start upsmon.

    systemctl enable nut-monitor.service # assuming the host is running systemd
    service nut-monitor start
  4. Check /var/log/daemon.log for a message indicating that upsmon was able to successfully connect to the UPS.
    You can also check that the server sees the client is connected by running

    upsc -c «upsname»@«upshost»
    And verifying that the IP address for the machine you are working on is in the list.

Other Notes

Configuration lives in /etc/nut.

Some aspects of configuration changed in NUT 2.0 (ships with wheezy and newer). Namely:

Another thing to note, is that wheezy appears to install brltty for some reason or another. It is EVIL and disconnects FTDI devices thinking they're braille terminals. You need to remove the "brltty" package in order to use the serial converters we bought for UPS monitoring.

How It Handles Issues

TODO: kbare—verify this is true.

When the battery level of the UPS becomes critical, the upsmon processes will notice, and initiate a system shutdown. The master will wait for all of the slaves to shut down, and then it will shut itself down, and power off the UPS.


CategoryServices CategoryClubServices

Services/Club UPS Monitoring (last edited 2022-10-01 16:28:25 by kbare@CLUB.CC.CMU.EDU)