[NOTIFICATIONS] Apache Tomcat upgrade from 9.0.118 to 9.0.120

Title

Cannot close yet: Close all activities before closing this item.

Status

Due

Scheduled

Context

Project

Category

Notes Saved

Supports Markdown: bold, italics, code, lists, links. Auto-saves.

AI Notes - managed by Claude

## Apache Tomcat 9.0.118 -> 9.0.120

Security upgrade to mitigate multiple vulnerabilities. Reference: https://tomcat.apache.org/security-9.html

Requested by Santiago Romero. Same procedure as ST #669 (9.0.117 -> 9.0.118).

**Rolling upgrade — one node at a time, validate between each.** All work runs from the ansible prod server `ansbl-p-mng-01.global.un.org`.

### Open questions / discrepancies in the procedure email

1. **Playbook filename is inconsistent** — the email uses three different paths:
   - `playbooks/jobs/notifications-tomcat-update-playbook.yml` (in the vim step)
   - `playbooks/jobs/notif-tomcat-update-playbook.yml` (first ansible-playbook run)
   - `playbooks/notif-tomcat-update-playbook.yml` (second ansible-playbook run, missing `jobs/`)

The KB runbook records the real name as `playbooks/jobs/notifications-update-tomcat.yml`. Confirm on the ansible host with `ls playbooks/jobs/ | grep -i tomcat` before running. This same error was flagged to Santiago on ST #669.

2. **PROD date not fixed.** Santiago asked for Thursday 24/07 because Eladio is on leave the following week. The KB runbook mandates a 3-day validation window after PRE. Decision: do PRE first, confirm PROD date with Santiago once PRE is validated.

---

### Prep (once, before PRE)

```
ssh adm.crodrigo2@ansbl-p-mng-01.global.un.org
sudo su -
cd /root/
bash get_ansible.sh
cd /root/ansible/
ls playbooks/jobs/ | grep -i tomcat
```

Confirm the exact playbook filename from that last command and use it for every run below.

---

## PRE

### 1. Edit the playbook — leave only notif-e-app-01

```
vim playbooks/jobs/notifications-update-tomcat.yml
```

```yaml
---
#-- NOTIFICATIONS playbook to update Tomcat
- hosts:
    ###- notif-d-app-01
    ###- notif-q-app-01
    ###- notif-d-app-02
    ###- notif-d-app-03
    ###- notif-u-app-01
    ###- notif-u-app-02
    - notif-e-app-01
    #- notif-e-app-02
    #- notif-p-app-01
    #- notif-p-app-02
  roles:
    - role: notifications_update_tomcat
```

### 2. Run the playbook

```
ansible-playbook playbooks/jobs/notifications-update-tomcat.yml
```

### 3. Validate notif-e-app-01

```
ssh root@notif-e-app-01
ps auxwww | grep tomcat | grep -v grep
ls -l /opt/tomcat
```

Expected symlink target:

```
lrwxrwxrwx 1 root root 25 /opt/tomcat -> /opt/apache-tomcat-9.0.120
```

### 4. Edit the playbook — swap to notif-e-app-02

```
vim playbooks/jobs/notifications-update-tomcat.yml
```

```yaml
---
#-- NOTIFICATIONS playbook to update Tomcat
- hosts:
    ###- notif-d-app-01
    ###- notif-q-app-01
    ###- notif-d-app-02
    ###- notif-d-app-03
    ###- notif-u-app-01
    ###- notif-u-app-02
    ###- notif-e-app-01
    - notif-e-app-02
    #- notif-p-app-01
    #- notif-p-app-02
  roles:
    - role: notifications_update_tomcat
```

### 5. Run the playbook again

```
ansible-playbook playbooks/jobs/notifications-update-tomcat.yml
```

### 6. Validate notif-e-app-02

```
ssh root@notif-e-app-02
ps auxwww | grep tomcat | grep -v grep
ls -l /opt/tomcat
```

### 7. Validate the service (PRE)

https://pre-comms.uniteaware.un.org/api/v1/rest/monitor

---

## PROD

Date to be confirmed with Santiago after PRE validation. HPSM change required before starting.

### 1. Refresh ansible

```
ssh adm.crodrigo2@ansbl-p-mng-01.global.un.org
sudo su -
cd /root/
bash get_ansible.sh
cd /root/ansible/
```

### 2. Edit the playbook — leave only notif-p-app-01

```
vim playbooks/jobs/notifications-update-tomcat.yml
```

```yaml
---
#-- NOTIFICATIONS playbook to update Tomcat
- hosts:
    ###- notif-e-app-01
    ###- notif-e-app-02
    - notif-p-app-01
    #- notif-p-app-02
  roles:
    - role: notifications_update_tomcat
```

### 3. Run the playbook

```
ansible-playbook playbooks/jobs/notifications-update-tomcat.yml
```

### 4. Validate notif-p-app-01

```
ssh root@notif-p-app-01
ps auxwww | grep tomcat | grep -v grep
ls -l /opt/tomcat
```

### 5. Edit the playbook — swap to notif-p-app-02

```
vim playbooks/jobs/notifications-update-tomcat.yml
```

```yaml
---
#-- NOTIFICATIONS playbook to update Tomcat
- hosts:
    ###- notif-e-app-01
    ###- notif-e-app-02
    ###- notif-p-app-01
    - notif-p-app-02
  roles:
    - role: notifications_update_tomcat
```

### 6. Run the playbook again

```
ansible-playbook playbooks/jobs/notifications-update-tomcat.yml
```

### 7. Validate notif-p-app-02

```
ssh root@notif-p-app-02
ps auxwww | grep tomcat | grep -v grep
ls -l /opt/tomcat
```

### 8. Validate the service (PROD)

https://comms.uniteaware.un.org/api/v1/rest/monitor

---

## Post-deployment cleanup

After PROD is fully validated, delete the old version logs on all four nodes (notif-e-app-01, notif-e-app-02, notif-p-app-01, notif-p-app-02):

```
rm -rf /opt/apache-tomcat-9.0.118/logs/
```

---

## Rollback

Per node, if the upgrade fails or the service does not come back:

```
cd /opt/
systemctl stop tomcat
rm tomcat
ln -s apache-tomcat-9.0.118 tomcat
systemctl start tomcat
systemctl status tomcat
```

Verify the symlink points back to 9.0.118 and the monitor endpoint responds before moving on.

📂 Activities for this Item

Drag using ⇅ to reorder; saves automatically.

➕ Add Activity

Open Activities

⇅

Upgrade Tomcat 9.0.120 on notif-p-app-01 (PROD) Next

Today: 0:00
⇅

[NOTIFICATIONS] Apache Tomcat upgrade from 9.0.118 to 9.0.120

Today: 1:45
⇅

Upgrade Tomcat 9.0.120 on notif-p-app-02 (PROD)

Today: 0:00
⇅

Cleanup old 9.0.118 logs on all 4 nodes

Today: 0:00
⇅

Inform requester (PROD done)

Today: 0:00

Closed Activities

Service Requests / Work Orders

RFS-1-13037077208

Service Request

WO-RFS-1-13037077213

Normalized time tracking

Last 30 days

Logged: 1:45 · Normalized: 0:00

2026-07-21

Normalized day not locked. Open day report

Day total: 0:00

Normalized (0) · 0:00

No normalized entries

Time entries 1:45

Manual log

[NOTIFICATIONS] Apache Tomcat upgrade from 9.0.118 to 9.0.120
Manual log

1:45 · 12:07

Meeting Notes (Project / Context)

Select a meeting to preview details.

iNeed Text

Title

Description

UA Notifications - [NOTIFICATIONS] Apache Tomcat upgrade from 9.0.118 to 9.0.120
Assign to Carlos Rodrigo - AOM DS

Asset#

Defined per context on the Taxonomy page.

Templates

Change Title

Change Description

Change Steps

Template Notes

Communications

Label

Order

Subject

Email Body

Attachments

No attachments yet.