Installing Anitya Upstream Release Monitoring on RHEL 7

Anitya is a release monitoring project developed and provided by the Fedora (Fedora) project. It is used to monitor upstream software projects for new releases in a automated fashion. It is thus similar to the uscan command from the Debian (Debian) devscripts package. Unlike the uscan command, Anitya is written in Python (Python) and comes with an integrated WebUI. Also unlike uscan, Anitya uses the FedMsg (Federated Message Bus) in order to publish messages indicating a detected upstream release event. These messages can in turn be consumed by other systems in order to trigger various actions upon a new upstream release. Anitya can be used “as a service” through https://release-monitoring.org/ or be installed as a dedicated instance. This article describes the necessary steps to set up an own, dedicated Anitya instance on a RHEL 7 system.

According to the official Anitya installation and configuration documentation there are only two steps – pip install anitya and editing /etc/anitya/anitya.toml – needed to set up Anitya. I have found this a bit too optimistic in case of a RHEL 7 system which was the installation target in my case. Here is a more detailed description of the necessary steps to get Anitya up and running on RHEL 7:

Install some basic Anitya dependencies like Python, the Python installer, the Apache webserver, the Apache WSGI module and some development files:
```
root@host:# yum install python httpd mod_wsgi python2-pip python-devel zeromq-devel
```
Install Anitya and its dependencies:
```
root@host:# pip2 install anitya
```
Unfortunately, the Python 2 environment is necessary since the Apache WSGI module in RHEL 7 is compiled against Python 2.

Adjust the import of the Anitya classes in the WGSI application. Open the file /usr/lib/python2.7/site-packages/anitya/wsgi.py in an editor of your choice:

root@host:# vi /usr/lib/python2.7/site-packages/anitya/wsgi.py

Alter its contents like shown in the following patch:

wsgi.py

--- /usr/lib/python2.7/site-packages/anitya/wsgi.py.orig        2018-12-05 16:59:26.000000000 +0100
+++ /usr/lib/python2.7/site-packages/anitya/wsgi.py     2018-12-05 16:40:40.000000000 +0100
@@ -14,7 +14,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
-from .app import create
+from anitya.app import create
 
 
 application = create()

Create and adjust the basic Anitya configuration.

Create the Anitya configuration directory:
```
root@host:# mkdir /etc/anitya
```
and download the sample configuration file from the Anitya GitHub repository:
```
root@host:# wget -O /etc/anitya/anitya.toml https://raw.githubusercontent.com/release-monitoring/anitya/master/files/anitya.toml.sample
```
Generate a secret key with e.g. the pwgen command:
```
root@host:# pwgen -sncB 32 1
```
Adjust the several settings in the Anitya configuration file. Open the file /etc/anitya/anitya.toml in an editor of your choice:
```
root@host:# vi /etc/anitya/anitya.toml
```
Alter the four configuration entries admin_email, secret_key, db_url and default_regex like shown in the following example:
/etc/anitya/anitya.toml
```
admin_email = "<YOUR-EMAIL-ADDRESS>"
secret_key = "<SECRET-KEY>"
db_url = "sqlite:////var/www/html/anitya.sqlite"
default_regex = '''(?i)%(name)s(?:[-_]?(?:minsrc|src|source))?[-_]([^-/_\s]+?)(?:[-_]
                   (?:minsrc|src|source|asc|release))?\.(?:tar|t[bglx]z|tbz2|zip)'''
```
Substitute the placeholder <YOUR-EMAIL-ADDRESS> with the email address at which you want to receive emails about errors of the Anitya application. This address will also be used as part of the request headers which identify the Anitya HTTP user agent. Substitute the placeholder <SECRET-KEY> with the secret key generated above.

In case a SQLite database is used, make sure the value of the db_url entry points to a location within the filesystem that is reachable from the context of the webserver the Anitya application is running in. If e.g. the Apache webserver is used, the SQLite database should be located somewhere within the DocumentRoot of the Apache VirtualHost.

Be careful with the syntax of the configuration file and the configuration entries in there. It seems that the configuration parsing part of Anitya is not very robust. A single invalid or syntactically incorrect entry will cause Anitya to skip all other entries as well and fall back to its default values. It took me a while to figure out that the Anitya database was generated and searched in the default location sqlite:////var/tmp/anitya-dev.sqlite no matter which value was given for the db_url configuration entry, just because another configuration entry – specifically the default_regex – had a syntactically incorrect value.

Download the Anitya cron job from the Anitya GitHub repository:

root@host:# cd /usr/lib/python2.7/site-packages/anitya
root@host:# wget https://raw.githubusercontent.com/release-monitoring/anitya/<RELEASE>/files/anitya_cron.py

The Anitya cron job should match the installed Anitya release. To ensure this, substitute the placeholder <RELEASE> in the above URL with the installed Anitya version. E.g. for the Anitya version 0.13.2:

root@host:# wget https://raw.githubusercontent.com/release-monitoring/anitya/0.13.2/files/anitya_cron.py

Make the Anitya cron job executeable:

root@host:# chmod 755 /usr/lib/python2.7/site-packages/anitya/anitya_cron.py

and adjust the Python interpreter to the RHEL 7 standard by opening the file /usr/lib/python2.7/site-packages/anitya/anitya_cron.py in an editor of your choice:

root@host:# vi /usr/lib/python2.7/site-packages/anitya/anitya_cron.py

Alter its contents like shown in the following patch:

anitya_cron.py.patch

# diff -uwi anitya_cron.py.orig anitya_cron.py
--- anitya_cron.py.orig 2018-12-23 19:52:31.000000000 +0100
+++ anitya_cron.py      2018-12-23 19:52:37.000000000 +0100
@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python2
 # -*- coding: utf-8 -*-
 
 import sys

Download the Anitya database creation and initialization script from the Anitya GitHub repository:
```
root@host:# cd /usr/lib/python2.7/site-packages/anitya
root@host:# wget https://raw.githubusercontent.com/release-monitoring/anitya/master/createdb.py
```
Run the downloaded script in order to create and initialize the Anitya database:
```
root@host:# python createdb.py
```
To grant access to the webserver the Anitya application is running in, adjust the ownership and the permissions of the Anitya database file:
```
root@host:# chown apache:apache /var/www/html/anitya.sqlite
root@host:# chmod 644 /var/www/html/anitya.sqlite
```

Reapply changes to the database from a missing database version.

Configure Alembic by opening the file /usr/lib/python2.7/site-packages/anitya/alembic.ini in an editor of your choice:

root@host:# cd /usr/lib/python2.7/site-packages/anitya
root@host:# vi alembic.ini

Insert the contents like shown in the following configuration example:

alembic.ini

# A generic, single database configuration.
 
[alembic]
# path to migration scripts
script_location = /usr/lib/python2.7/site-packages/anitya/db/migrations
 
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
 
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
 
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
 
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
sqlalchemy.url = sqlite:////var/www/html/anitya.sqlite
 
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
 
[handlers]
keys = console
 
[formatters]
keys = generic
 
[logger_root]
level = WARN
handlers = console
qualname =
 
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
 
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
 
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

Initialize the Alembic table in the Anitya database and show the current database version:

root@host:# alembic current
7a8c4aa92678 (head)

Manually change the database version known to Alembic in order to trick it into again applying the missing change to the database:

root@host:# sqlite3 /var/www/html/anitya.sqlite

sqlite> INSERT INTO alembic_version (version_num) VALUES ('a52d2fe99d4f');
sqlite> .quit

Reapply the change to the database from the missing database version feeaa70ead67:

root@host:# alembic upgrade feeaa70ead67

Determine the current “head” database version and manually change the database version known to Alembic:

root@host:# alembic history
540bdcf7edbc -> 7a8c4aa92678 (head), Add missing GitHub owner/project pairs
27342bce1d0f -> 540bdcf7edbc, Convert GitHub URL to owner/project
[...]

root@host:# sqlite3 /var/www/html/anitya.sqlite

sqlite> UPDATE alembic_version SET version_num = '7a8c4aa92678';
sqlite> .quit

Setup the Anitya cron job with the previously downloaded script by opening the file /etc/cron.d/anitya in an editor of your choice:
```
root@host:# vi /etc/cron.d/anitya
```
Insert the contents like shown in the following example:
/etc/cron.d/anitya
```
*/5 * * * * root /usr/lib/python2.7/site-packages/anitya/anitya_cron.py 1>>/var/log/anitya.log 2>&1
```
Adjust the execution interval according to your environment and needs.

Create a VirtualHost configuration for e.g. the Apache webserver in whose context the Anitya application will be running in and start the Apache webserver.

Create the VirtualHost configuration by opening the file /etc/httpd/conf.d/anitya.conf in an editor of your choice:

root@host:# vi /etc/httpd/conf.d/anitya.conf

Insert the contents like shown in the following configuration example:

/etc/httpd/conf.d/anitya.conf

<VirtualHost <IP>:<PORT>>
  DocumentRoot "/var/www/html/"
  ServerName <FQDN>
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
 
  WSGIDaemonProcess anitya user=apache group=apache processes=2 threads=25 python-path=/usr/lib/python2.7/site-packages
  WSGIProcessGroup anitya
  WSGIScriptAlias /anitya /usr/lib/python2.7/site-packages/anitya/wsgi.py process-group=anitya
 
  <Directory "/usr/lib/python2.7/site-packages/anitya">
    <Files wsgi.py>
      Order deny,allow
      Allow from all
      Require all granted
    </Files>
 
    Order allow,deny
    Options Indexes FollowSymLinks
    Allow from all
  </Directory>
 
  Alias "/anitya/static" "/usr/lib/python2.7/site-packages/anitya/static"
  <Directory "/usr/lib/python2.7/site-packages/anitya/static">
    Order allow,deny
    Options Indexes FollowSymLinks
    Allow from all
    Require all granted
  </Directory>
</VirtualHost>

Substitute the placeholders <IP> and <PORT> with the IP address and TCP port on which the virtual host should be listening. Substitute the placeholder <FQDN> with the fully qualified domain name under which you want to reach the Anitya application with your browser.

Enable and start the Apache webserver:

root@host:# systemctl enable httpd
root@host:# systemctl start httpd

After performing the steps outlined above, your own, dedicated Anitya instance on a RHEL 7 system should be ready. The WebUI of the Anitya instance will be reachable with a browser under the following URL http://<FQDN>/anitya, where the placeholder <FQDN> has to be substituted with the fully qualified domain name given in the above Apache VirtualHost configuration.

After logging into Anitya and adding an upstream software project, the next run of the Anitya cron job should discover the initial and current versions of the upstream software project. This should be visible in two places, firstly in the logfile of the Anitya cron job /var/log/anitya.log and secondly on the projects page of the Anitya WebUI. If this is not the case after the next execution interval of the Anitya cron job, review the logfile /var/log/anitya.log for possible errors.

I hope you find the provided Anitya installation and configuration guide for a RHEL 7 target system useful and enjoyed reading this blog post. Please don't hesitate to drop me a note if you have any suggestions or run into any issues with this guide. In future articles i will – hopefully soon – write about the neccessary steps to set up the FedMsg (Federated Message Bus) and connect Anitya to it, as well as an enhancement to Anitya itself.

0 Comments | 2019-01-06 written by Frank Fegert (XING) | Permanentlink
Tags:

2017-08-05 // In-place Upgrade from RHEL 6 to RHEL 7 with separate /usr Filesystem

Officially there is no support for an in-place upgrade from RHEL 6 to RHEL 7 within a Hyper-V VM and with /usr residing on a separate filesystem. The latter issue is described in the Red Hat knowledge base article Why does Red Hat Enterprise Linux 7 in-place upgrade fails if /usr is on separate partition?. With several tweaks in the right places an in-place upgrade with /usr residing on a separate filesystem is still possible. This article describes the necessary steps.

Disclaimer: This guide comes with no fitness for purpose or guarantee whatsoever! I'm not liable for any damage resulting from following the described procedure. Before starting an upgrade ensure that you understand the steps to be performed and have arranged a service downtime as well as a proper fallback scenario and have a working backup in place!

Create a backup and or a snapshot of the system to be upgraded.
Follow the procedure outlined in How do I upgrade from Red Hat Enterprise Linux 6 to Red Hat Enterprise Linux 7?.

The preupgrade assistant will fail with at least the following messages:

root@rhel6:# preupg

The Preupgrade Assistant is a diagnostics tool
and does not perform the actual upgrade.
Do you want to continue? [Y/n]
y
Gathering logs used by the Preupgrade Assistant:

[...]

|In-place upgrade requirements for the /usr/ directory                        |fail              |
|Hyper-V                                                                      |fail              |
--------------------------------------------------------------------------------------------------
The tarball with results is stored in '/root/preupgrade-results/preupg_results-170731111749.tar.gz' .
The latest assessment is stored in the '/root/preupgrade' directory.
Summary information:
We have found some critical issues. In-place upgrade or migration is not advised.
Read the file /root/preupgrade/result.html for more details.
Read the admin report file /root/preupgrade/result-admin.html for more details.
Please ensure you have backed up your system and/or data
before doing a system upgrade to prevent loss of data in
case the upgrade fails and full re-install of the system
from installation media is needed.
Upload results to UI by the command:
e.g. preupg -u http://example.com:8099/submit/ -r /root/preupgrade-results/preupg_results-170731111749.tar.gz .

Make sure any other issues or conflicts are resolved and re-run the the preupgrade assistant iteratively until all but the two issues shown above are resolved.

Force starting the upgrade process by ignoring the last two remaining issues shown above:
```
root@rhel6:# redhat-upgrade-tool -f --network <VERSION> --instrepo <URL>
```
After the upgrade is finished do not reboot immediately!
Modify the GRUB configuration in /boot/grub/grub.conf first. This is necessary in order to activate the logical volume that contains the /usr filesystem at boot time:
```
root@rhel6:# vi /boot/grub/grub.conf
```
In the section title System Upgrade (redhat-upgrade-tool) add a rd.lvm.lv=VG/LV stanza to the kernel line. E.g. for the logical volume lv_usr within the volume group vg00:
```
[...]
title System Upgrade (redhat-upgrade-tool)
        root (hd0,0)
        kernel /vmlinuz-redhat-upgrade-tool [...] rd.lvm.lv=vg00/lv_usr [...]
        initrd /initramfs-redhat-upgrade-tool.img
[...]
```
Modify the system init script /etc/rc.d/rc.sysinit in order to prevent file system checks at boot time. Otherwise the startup of the upgrade system will fail, due to the fact that /usr is now already mounted from the boot stage.
```
root@rhel6:# vi /etc/rc.d/rc.sysinit
```
Comment the fsck command in line 422 of the init script /etc/rc.d/rc.sysinit, like shown in the following example:
/etc/rc.d/rc.sysinit
```
420         STRING=$"Checking filesystems"
421         echo $STRING
422         #fsck -T -t noopts=_netdev -A $fsckoptions
423         rc=$?
424
```
The line numbers are only shown for purpose of illustration. Do not insert the line numbers at the beginning of the line!
Reboot the system to start the actual upgrade process.
After the upgrade to RHEL 7 has finished, the system might hang on the post-upgrade reboot due to the logical volume containing the /usr filesystem not being available. In order to circumvent this issue, stop the automatic boot of the upgraded system and perform a manual boot with modified GRUB parameters.
- In the GRUB menu navigate to the entry Red Hat Enterprise Linux Server ([…]) and press E to enter the command line editing mode.
- Navigate to the kernel /vmlinuz[…] entry and again press E.
- Like above, add a rd.lvm.lv=VG/LV stanza for the logical volume containing the /usr filesystem to the kernel line. E.g. for the logical volume lv_usr within the volume group vg00 add the stanza rd.lvm.lv=vg00/lv_usr.
- Press RETURN followed by B to start the boot of the RHEL 7 system.

After the RHEL 7 system has sucessfully booted, don't forget to permanently add the changes to the GRUB command line from the previous step to the GRUB configuration /boot/grub/grub.conf:

root@rhel7:# vi /boot/grub/grub.conf

For every title section add a rd.lvm.lv=VG/LV stanza to the kernel line. E.g. for the logical volume lv_usr within the volume group vg00:

[...]
title Red Hat Enterprise Linux Server 7.3 Rescue 3b8941354040abfa3882f5ca00000039 (3.10.0-514.el7.x86_64)
        [...]
        kernel /vmlinuz-0-rescue-[...] rd.lvm.lv=vg00/lv_usr [...]
        [...]
title Red Hat Enterprise Linux Server (3.10.0-514.el7.x86_64) 7.3 (Maipo)
        [...]
        kernel /vmlinuz-[...] rd.lvm.lv=vg00/lv_usr [...]
        [...]
[...]

Finish the procedure outlined in How do I upgrade from Red Hat Enterprise Linux 6 to Red Hat Enterprise Linux 7?.

All done, your system should now have successfully been upgraded from RHEL 6 to RHEL7.

1 Comment | 2017-08-05 written by Frank Fegert (XING) | Permanentlink
Tags:

2019-01-06 // Installing Anitya Upstream Release Monitoring on RHEL 7

2017-08-05 // In-place Upgrade from RHEL 6 to RHEL 7 with separate /usr Filesystem

Recent Comments

Syndiaction / RSS Feed

A word from our sponsors

QR Code: URL of current page

2019-01-06 // Installing Anitya Upstream Release Monitoring on RHEL 7

2017-08-05 // In-place Upgrade from RHEL 6 to RHEL 7 with separate /usr Filesystem

Tag Cloud

Recent Comments

Blogroll

Syndiaction / RSS Feed

A word from our sponsors

QR Code: URL of current page