Katello: Import CentOS Errata into Pulp

Working with Katello – part 2. We will be importing CentOS errata into Pulp.

This article is part of the Homelab Project with KVM, Katello and Puppet series.

Homelab

We have Katello installed on a CentOS 7 server:

katello.hl.local (10.11.1.4) – see here for installation instructions

See the image below to identify the homelab part this article applies to.

Credits

This is possible thanks to the following sites:

  1. https://cefs.steve-meier.de/ – CEFS: CentOS Errata for Spacewalk
  2. https://github.com/rdrgmnzs/pulp_centos_errata_import – the script that imports CentOS errata into Pulp/Katello

Installation

Install the following packages:

# yum install git \
  pulp-admin-client \
  pulp-rpm-admin-extensions \
  pulp-rpm-consumer-extensions \
  pulp-rpm-handlers \
  pulp-rpm-yumplugins \
  pulp-rpm-admin-extensions \
  pulp-consumer-client \
  python-pulp-agent-lib \
  perl-Text-Unidecode \
  perl-XML-Simple \
  perl-XML-Parser

Configure CentOS Errata Import

Make sure that the CentOS repositories have been synced. We’ve done this in the previous article.

Clone the repository:

# cd /opt && git clone https://github.com/rdrgmnzs/pulp_centos_errata_import.git
# cd ./pulp_centos_errata_import

Download the latest errata archive and extract the file:

# wget -N https://cefs.steve-meier.de/errata.latest.xml.bz2
# bunzip2 ./errata.latest.xml.bz2

Configure Pulp authetication:

# mkdir -m0700 ~/.pulp
# cat /etc/pki/katello/certs/pulp-client.crt /etc/pki/katello/private/pulp-client.key > ~/.pulp/user-cert.pem
# chmod 0400 ~/.pulp/user-cert.pem

Import CentOS Errata into Pulp

Chances are that we won’t know which repository IDs to use when importing CentOS errata, therefore we can run the following command without specifying any (it may take longer):

# perl ./errata_import.pl --errata=errata.latest.xml

Alternatively, we can view a list of Pulp repositories by using following command:

# pulp-admin repo list | less

And then import errata into repositories required, e.g.:

# perl ./errata_import.pl \
  --errata=errata.latest.xml \
  --include-repo=83418f8f-e069-4b24-8e67-4b07af5e87d2 \
  --include-repo=0abbd033-67cc-4e42-bff0-237d52f1bcdb

There was a case when we got the error saying “Skipping errata […] — No packages found”. If that happens, try passing the repository ID you want to load the errata for with the –include-repo flag.

After importing errata, use the mighty Hammer to force post-sync actions (Katello 3.0 and up):

# hammer settings set \
  --name "force_post_sync_actions" \
  --value "true"

We have to sync the repositories so that errata is published (and visible in the Katello interface).

# hammer repository synchronize \
  --name "base_x86_64" \
  --product "el7_repos"

If all goes well, we should see some errata:

# hammer repository info \
  --name "base_x86_64" \
  --product "el7_repos"
ID:                 1
Name:               base_x86_64
Label:              base_x86_64
Organisation:       Lisenet
Red Hat Repository: no
Content Type:       yum
Checksum Type:      sha256
Mirror on Sync:     no
URL:                http://mirror.centos.org/centos/7/os/x86_64/
Publish Via HTTP:   yes
Published At:       http://katello.hl.local/pulp/repos/lisenet/Library/custom/el7_repos/base_x86_64/
Relative Path:      lisenet/Library/custom/el7_repos/base_x86_64
Download Policy:    on_demand
Product:            
    ID:   1
    Name: el7_repos
GPG Key:            
    ID:   1
    Name: RPM-GPG-KEY-CentOS-7
Sync:               
    Status:         Success
    Last Sync Date: 28 days
Created:            2018/02/20 20:56:52
Updated:            2018/02/21 23:16:44
Content Counts:     
    Packages:       9591
    Package Groups: 84
    Errata:         830

Note how “Mirror on Sync” is set to “no”. This is to prevent loaded errata from being cleared on next sync.

Here is some info for a randomly selected errata ID:

# hammer erratum info --id 4191
ID:          4191
Errata ID:   CESA-2018:0260
Title:       CentOS systemd Security Update
Type:        security
Severity:    Moderate
Issued:      2018-02-01
Updated:     2018-02-01
Description: Not available
Summary:     
Solution:

17 thoughts on “Katello: Import CentOS Errata into Pulp

  1. I’m still having issues with importing,

    It seems that it imported ~4k erratas for EPEL, but nothing for the others (especially base)

    running it displays a whole bunch of :

    NOTICE: Skipping errata CEBA-2018:0420 (CentOS openscap BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0421 (CentOS xorg-x11-server BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0422 (CentOS nfs-utils BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0510 (CentOS sssd BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0511 (CentOS ksh BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0513 (CentOS gcc BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0514 (CentOS irqbalance BugFix Update) -- No packages found
    NOTICE: Skipping errata CEBA-2018:0597 (CentOS tzdata BugFix Update) -- No packages found
    INFO: Errata for CEBA-2018:0658 already exists
    INFO: Errata for CEBA-2018:0659 already exists
    INFO: Errata for CEBA-2018:0661 already exists
    INFO: Errata for CEBA-2018:0662 already exists
    INFO: Errata for CEBA-2018:0663 already exists
    INFO: Errata for CEBA-2018:0664 already exists
    INFO: Errata for CEBA-2018:0665 already exists
    INFO: Errata for CEBA-2018:0668 already exists
    INFO: Errata for CEBA-2018:0669 already exists

    followed by another whole bunch of (which take a VERY LONG TIME):

    INFO: Creating errata for CEBA-2018:0728 (CentOS python-slip BugFix Update) (3 of 3)
    INFO: +----------------------------------------------------------------------+
                                  Unit Upload
    +----------------------------------------------------------------------+
    
    Extracting necessary metadata for each request...
    ... completed
    
    Creating upload requests on the server...
    [==================================================] 100%
    Initializing upload
    ... completed
    
    Starting upload of selected units. If this process is stopped through ctrl+c,
    the uploads will be paused and may be resumed later using the resume command or
    canceled entirely using the cancel command.
    
    Importing into the repository...
    This command may be exited via ctrl+c without affecting the request.
    
    
    [\]
    Running...
    
    Task Succeeded
    
    
    Deleting the upload request...
    ... completed

    Command I’m trying now is:

    # perl ./errata_import.pl --errata=errata.latest.xml --include-repo=c659ccfb-afeb-431c-891e-3228e683008e

    the ID is for base repo. I tried without it as well before that which gave me 4k erratas on EPEL.

    Any ideas?

    Thanks

    • I think there is some confusion here. The script imports CentOS errata, but it has nothing to do with EPEL errata. While EPEL repository provides security metadata, CentOS repositories don’t supply it. This is the reason why we want to import CentOS errata from CentOS-Announce mailing list.

      From the logs that you’ve posted, it looks like CentOS errata has already been applied. Try forcing post-sync actions and then re-sync the repository.

    • That’s why I was confused too. Right now I see available Errata like “FEDORA-EPEL-2018-ad387c7768” in my system but nothing under base repo.

      Once I applied force post sync actions and did repo sync, I started seeing base repo errata.

      By the way thanks so much for this series. I’m learning a lot from it.

  2. Conflicts for Katello 3.7 when installing the packages
    ——————-
    –> Processing Conflict: katello-agent-3.3.4-1.el7.noarch conflicts pulp-consumer-client
    –> Finished Dependency Resolution
    Error: katello-agent conflicts with pulp-consumer-client-2.16.4-1.el7.noarch
    You could try using –skip-broken to work around the problem
    You could try running: rpm -Va –nofiles –nodigest

    • I used Katello 3.6 and didn’t have this problem I’m afraid. I later upgraded Katello from 3.6 to 3.7 and errata routine still worked. Also, I don’t have the katello-agent package installed on the server.

      It looks like some packages are obsoleted by katello-agent: pulp-rpm-handlers and python-pulp-agent-lib. This may very well be the case here.

    • @Eric, et al.:

      As of today, running the “yum” command, verbatim, as shown in the article, results in the following error:

      # yum install ...
      
      --> Processing Conflict: katello-agent-3.3.5-1.el7.noarch conflicts pulp-consumer-client
      --> Finished Dependency Resolution
      --> Finding unneeded leftover dependencies
      Found and removing 0 unneeded dependencies
      Error: katello-agent conflicts with pulp-consumer-client-2.16.4-1.el7.noarch
       You could try using --skip-broken to work around the problem
       You could try running: rpm -Va --nofiles --nodigest

      This is due to various changes since this article was first posted. In order to complete this step successfully, simply remove “pulp-rpm-handlers” and “python-pulp-agent-lib” from the list of packages to be installed; i.e.:

      # yum install git pulp-admin-client pulp-rpm-admin-extensions \
          pulp-rpm-consumer-extensions pulp-rpm-yumplugins pulp-consumer-client \
          perl-Text-Unidecode perl-XML-Simple perl-XML-Parser

      Additionally, you should be able to remove “git” from the package list as it is likely already installed by this point (if you have been following the previous articles).

      (N.B.: The comment form likely messed up the formatting of the above; hopefully you can figure it out!)

      @Tomas:

      Thanks for the great series of articles!

    • Thanks for your feedback!

      Try removing pulp-rpm-handlers and python-pulp-agent-lib from the list of packages to be installed as per suggestion in another comment.

  3. Very nice article!

    I am on 3.8 and it looks like force_post_sync_actions is no longer there..

    $ hammer settings list | grep force_post_sync_actions
    $ hammer settings set --name "force_post_sync_actions" --value "true"      
    Could not update the setting:
      Error: setting not found.
    • Thanks for your feedback. I’ve got several comments saying that there have been some changes between Katello versions. This is unfortunate but inevitable, because the project is in active development.

      My homelab is on Katello 3.6. According to this changelog, force_post_sync_actions has been deprecated, and has since been replaced by the skip-metadata-check hammer flag.

      Users have to set skip-metadata-check to force post-sync actions.

    • Hi,
      Did you ever get this working? I am having troubles importing errata also.
      I’ve tried the following but still no luck:
      hammer repository synchronize –skip-metadata-check true –id 3
      i’ve also tried to recreate the repository using: –mirror-on-sync=false to prevent loaded errata from being cleared on next sync but no luck either. Thanks.
      Phat

  4. Hi,
    I am on 3.8 and tried to use the ‘–skip-metadata-check true’ to force post-sync actions, however it does not seem to be getting the errata data – any idea?

    hammer repository synchronize –product ‘Centos 7’ –name ‘Centos 7 – Base – x86_64’ –skip-metadata-check true –organization MyOrg

    hammer repository info –id 1
    ID: 1
    Name: Centos 7 – Base – x86_64
    Label: Centos_7_-_Base_-_x86_64
    Organization: MyOrg
    Red Hat Repository: no
    Content Type: yum
    Mirror on Sync: yes
    URL: http://mirror.centos.org/centos/7/os/x86_64/
    Publish Via HTTP: yes
    Published At: http://katello****/pulp/repos/Vendavo/Library/custom/Centos_7/Centos_7_-_Base_-_x86_64/
    Relative Path: MyOrg/Library/custom/Centos_7/Centos_7_-_Base_-_x86_64
    Download Policy: on_demand
    Product:
    ID: 1
    Name: Centos 7
    GPG Key:

    Sync:
    Status: Success
    Last Sync Date: less than a minute
    Created: 2018/11/13 20:54:23
    Updated: 2018/11/13 21:09:36
    Content Counts:
    Packages: 9911
    Source RPMS: 0
    Package Groups: 88
    Errata: 0

  5. Sorry, please disregard my previous comment, apparently all errata appeared after disabling ‘Mirror on Sync’.

    • Thanks. Yes, as mentioned in the blog post, “Mirror on Sync” needs setting to “no” in order to prevent loaded errata from being cleared on next sync

Leave a Reply

Your email address will not be published. Required fields are marked *