Set up Spacewalk 2.4 on CentOS 6

Part 6 of setting up a Linux home lab environment with VirtualBox. Check this blog post for more info.

Spacewalk is an open source Linux systems management solution. It is the upstream community project from which the Red Hat Satellite products is derived.

Software

Software used in this article:

1. CentOS 6
2. Spacewalk 2.4
3. PostgreSQL 8.4

System Setup

1. CentOS 6 (x86_64) minimal, fully up-to-date,
2. 10GB of free disk space on / with
3. /var/satellite mounted on a dedicated logical volume, with 50GB of disk space, and
4. /var/lib/pgsql mounted on a dedicated logical volume, with 5GB of disk space,
5. SELinux set to enforcing mode,
6. Iptables turned on with inbound TCP ports 80 and 443 open for Spacewalk WebUI,
7. A fully qualified domain name (FQDN) spacewalk.hl.local.

Setup Spacewalk Server with PostgreSQL Backend

Configure Repositories

Configure required repositories for Spacewalk. EPEL is needed for dependencies (jabberd, dojo etc).

# yum install http://yum.spacewalkproject.org/2.4/RHEL/6/x86_64/spacewalk-repo-2.4-3.el6.noarch.rpm
# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm

Import Spacewalk’s public key:

# rpm --import http://yum.spacewalkproject.org/RPM-GPG-KEY-spacewalk-2015

Additional dependencies are needed from jpackage.

# cat << EOL >/etc/yum.repos.d/jpackage-generic.repo
[jpackage-generic]
name=JPackage generic
mirrorlist=http://www.jpackage.org/mirrorlist.php?dist=generic&type=free&release=5.0
enabled=1
gpgcheck=1
gpgkey=http://www.jpackage.org/jpackage.asc
EOL

It is recommended to make the JPackage Project GPG key available to RPM:

# rpm --import http://www.jpackage.org/jpackage.asc

Install Spacewalk Packages

# yum install -y spacewalk-setup-postgresql spacewalk-postgresql

Configure PostgreSQL Server

We are going to run spacewalk-setup-postgresql to configure the PostgreSQL server. We will use the following data to configure the database server:

1. Database name: spaceschema
2. Database username: spaceuser
3. Database user password: changeme

If paranoid (highly optional):

# unset HISTFILE

Database parameters to pass on a command line (change as needed):

# DBNAME=spaceschema; DBUSER=spaceuser; DBPASSWORD=changeme

Configure Spacewalk database:

# spacewalk-setup-postgresql create \
 --db $DBNAME \
 --user $DBUSER \
 --password $DBPASSWORD

If you get the following error while setting it up:

createdb: database creation failed: ERROR:  new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)

open /usr/bin/spacewalk-setup-postgresql file for editing, change the line containing UTF8 to SQL_ASCII:

runuser - postgres -c "createdb -E UTF8 '$PGNAME'"

and re-run the database setup again. When finished, test the database connection:

# PGPASSWORD=changeme psql -a -U spaceuser spaceschema
psql (8.4.20)
Type "help" for help.

spaceschema=# \q

# PGPASSWORD=changeme psql -h localhost -a -U spaceuser spaceschema
psql (8.4.20)
Type "help" for help.

spaceschema=# \q

Setup Spacewalk

Setup Spacewalk, but skip embedded database installation:

# spacewalk-setup --disconnected --skip-db-install
* Setting up SELinux..
** Database: Setting up database connection for PostgreSQL backend.
** Database: Embedded database installation SKIPPED.
Could not connect to the database.  Your connection information may be incorrect.

Hostname (leave empty for local)?
Database? spaceschema
Username? spaceuser
Password? ******
** Database: Populating database.
*** Progress: #############################
* Configuring tomcat.
* Setting up users and groups.
** GPG: Initializing GPG and importing key.
** GPG: Creating /root/.gnupg directory
You must enter an email address.
Admin Email Address? root@localhost
* Performing initial configuration.
* Activating Spacewalk.
** Loading Spacewalk Certificate.
** Verifying certificate locally.
** Activating Spacewalk.
* Configuring apache SSL virtual host.
Should setup configure apache's default ssl server for you (saves original ssl.conf) [Y]?
** /etc/httpd/conf.d/ssl.conf has been backed up to ssl.conf-swsave
* Configuring jabberd.
* Creating SSL certificates.
CA certificate password? **********
Re-enter CA certificate password? **********
Organization? CentOS6
Organization Unit [spacewalk.hl.local]? Spacewalk
Email Address [root@localhost]?
City? Birmingham
State? West Midlands
Country code (Examples: "US", "JP", "IN", or type "?" to see a list)? GB
** SSL: Generating CA certificate.
** SSL: Deploying CA certificate.
** SSL: Generating server certificate.
** SSL: Storing SSL certificates.
* Deploying configuration files.
* Update configuration in database.
* Setting up Cobbler..
Cobbler requires tftp and xinetd services be turned on for PXE provisioning functionality. Enable these services [Y]?
* Restarting services.
Installation complete.
Visit https://spacewalk.hl.local to create the Spacewalk administrator account.

Enable and restart the Spacewalk service:

# /usr/sbin/spacewalk-service enable ;\
 /usr/sbin/spacewalk-service restart

We want to make RPM-GPG-KEY-spacewalk-2015 key available for clients with no Internet access:

# wget http://yum.spacewalkproject.org/RPM-GPG-KEY-spacewalk-2015 \
 -O /var/www/html/pub/RPM-GPG-KEY-spacewalk-2015

Spacewalk clients with LAN-only access can download the key from https://spacewalk.hl.local/pub/RPM-GPG-KEY-spacewalk-2015.

Check Spacewalk service status, make sure everything is running:

# /usr/sbin/spacewalk-service status
postmaster (pid  1371) is running...
router (pid 1615) is running...
sm (pid 1623) is running...
c2s (pid 1631) is running...
s2s (pid 1639) is running...
tomcat6 (pid 1570) is running... [  OK  ]
httpd (pid  1585) is running...
osa-dispatcher (pid  1655) is running...
rhn-search is running (1685).
cobblerd (pid 1740) is running...
RHN Taskomatic is running (1771).

Troubleshooting

When changing a postgresql password for the spacewalk user, do not forget to update the /etc/rhn/rhn.conf file with new credentials as Spacewalk backend may fail to connect successfully to the database:

When using /var/lib/pgsql on a dedicated logical volume with SELinux in enforcing mode, we may get the following error:

postmaster cannot access the server configuration file "/var/lib/pgsql/data/postgresql.conf": Permission denied

Check SELinux logs to confirm:

# grep postgres /var/log/audit/audit.log|grep denied|tail

To fix, add SELinux context mapping and apply to the running system:

# yum -y install policycoreutils-python
# semanage fcontext -a -t postgresql_db_t "/var/lib/pgsql(/.*)?"
# restorecon -Rv /var/lib/pgsql

Check logs:

# tail /var/log/messages
# tail /var/log/rhn/*.log

Spacewalk Repositories and Channels

Create a CentOS Base Repository

[UPDATE 2016]: CentOS 6.7 is deprecated, therefore you should use /6/ and not /6.7/ in your path.

Go to Channels -> Manage Software Channels -> Manage Repositories -> Create Repository.

Create a Parent Channel

Go to Channels -> Manage Software Channels -> Create Channel.

Create a Child Channel for CentOS Base

Go to Channels -> Manage Software Channels -> Create Channel.

Assing the CentOS Base Repository to the Child Channel

Go to Channels -> Manage Software Channels -> select the created channel -> Repositories -> select the previously created repository -> Update repositories.

Once updated, click on the Sync tab and then click on the Sync Now button to start package synchronisation from the upstream server. Once the package synchronisation is completed, you should see a number of available packages.

Below is a full list of channels that we have configured:

# spacewalk-repo-sync --list
======================================
|   Channel Label   |   Repository   |
======================================
epel-x86_64 | http://dl.fedoraproject.org/pub/epel/6/x86_64/
centos-6-os-x86_64 | http://mirror.centos.org/centos/6/os/x86_64/
spacewalk-latest-x86_64 | http://yum.spacewalkproject.org/latest/RHEL/6/x86_64/
puppetlabs-el6-x86_64 | http://yum.puppetlabs.com/el/6/products/x86_64/
elasticsearch-1.7 | http://packages.elastic.co/elasticsearch/1.7/centos
mongo-stable-x86_64 | http://repo.mongodb.org/yum/redhat/6/mongodb-org/stable/x86_64/
centos-6-updates-x86_64 | http://mirror.centos.org/centos/6/updates/x86_64/
puppetlabs-dep-el6-x86_64 | http://yum.puppetlabs.com/el/6/dependencies/x86_64/
spacewalk-client-latest-x86_64 | http://yum.spacewalkproject.org/latest-client/RHEL/6/x86_64/
centos-6-parent-x86_64 | No repository set

To sync, for example, the epel channel from a command line, do:

# spacewalk-repo-sync --channel epel-x86_64 --type yum

A simple script to sync all channels one by one:

# cat << EOL > /usr/local/bin/spacewalk-sync.sh
#!/bin/bash
CMD="/usr/bin/spacewalk-repo-sync ";
SYNC_DIR="/var/satellite";

# A dirty way of getting volume space usage
usage=$(df "$SYNC_DIR"|grep satellite|grep -o "...%"|cut -b1-3);
if [ "$usage" -gt "90" ]; then
 echo "WARNING: "$SYNC_DIR" usage is "$usage"%.";
 echo "This does not prevent the script from running.";
else
 echo ""$SYNC_DIR" usage looks OK "$usage"%.";
fi

# Low capacity repositories at the top
$CMD -c elasticsearch-1.7 -t yum;
$CMD -c spacewalk-client-latest-x86_64 -t yum;
$CMD -c mongo-stable-x86_64 -t yum;
$CMD -c puppetlabs-dep-el6-x86_64 -t yum;
$CMD -c spacewalk-latest-x86_64 -t yum;
$CMD -c centos-6-updates-x86_64 -t yum;
$CMD -c puppetlabs-el6-x86_64 -t yum;
$CMD -c centos-6-os-x86_64 -t yum;
$CMD -c epel-x86_64 -t yum;
exit 0;
EOL

Spacewalk Activation Keys

Go to Systems -> Activation Keys -> Create Key.

After creating the key, go to Child Channels tab and tick all channels. Any system registered using this activation key will be subscribed to the selected child channels.

Upload RPM Packages to Spacewalk

There are cases when you may need to push one or two customised RPMs to Spacewalk.

This can be done with rhnpush by specifying the channel the packages need to go to, for example:

# rhnpush -v --channel=checkinstall-el6 --server=http://localhost/APP --dir=packages
Connecting to http://localhost/APP
Username: admin
Password:
Package /root/packages/checkinstall-1.6.2-20.2.x86_64.rpm Not Found on RHN Server -- Uploading
Uploading package /root/packages/checkinstall-1.6.2-20.2.x86_64.rpm
Using POST request

Setup Spacewalk Client

Add a yum repository for Spacewalk as well as a matching EPEL repository:

# rpm -Uvh http://yum.spacewalkproject.org/2.4-client/RHEL/6/x86_64/spacewalk-client-repo-2.4-3.el6.noarch.rpm
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm

Install Spacewalk client packages:

# yum install rhn-client-tools rhn-check rhn-setup rhnsd m2crypto yum-rhn-plugin

Install Spacewalk’s CA certificate on the server to enable SSL communication:

# wget -O /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT http://spacewalk.hl.local/pub/RHN-ORG-TRUSTED-SSL-CERT

Register a client system with Spacewalk using the activation key that we have created earlier:

# cat /usr/local/bin/spacewalk-register.sh
#!/bin/bash

if [[ $# -gt 0 ]]; then
    reg_opts="$*"
fi

if [[ $reg_opts =~ --(activationkey|profilename) ]]; then
    echo "usage: $0 "
    echo " are passed as is to rhnreg_ks"
    echo "       do not use profile name or activationkey opts these are"
    echo "       hardcoded into the script"
    exit 1
fi

read -p "Enter profile name: " profile
echo "Registering system as: ${profile}"
rhnreg_ks --force --activationkey="1-centos6-custom-spacewalk-key" \
  --serverUrl=http://spacewalk.hl.local/XMLRPC \
  --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT \
  --profilename=${profile} ${reg_opts}