OSPF Routes Missing after Master Engine Cluster Node Reboot

OSPF Routes Missing after Master Engine Cluster Node Reboot

Summary
OSPF routes from Virtual Engines might be missing after Master Engine Cluster node reboot when one of VEs is central OSPF router to whom other VEs talk to.
Notes and Warnings

Note Quagga is used for dynamic routing operations in NGFW version 6.10, while the following major versions moved into using FRR. This article applies only to Quagga.

Problem

Master Engine cluster with dozens of Virtual Engines (VE). All VEs run quagga OSPF with one VE as the central OSPF process all other VEs talk to.

Normally in OSPF graceful restart, one OSPF router restarts and all adjacent routers work in graceful restart helper mode. When on Master Engine cluster one node is rebooted, as example due to node upgrade, half of OSPF routers restart at same time. This is challenge for graceful operation and likely leads to some routes missing and not recovering until OSPF process is restarted.

Solution

Forcepoint recommends setup like this with one central OSPF process exchanging routes with all other VEs to run without graceful restart option.

Starting from NGFW 6.10.16 version is available tuning option to disable graceful restart. To apply it:

  1. Create empty tuning file on Master cluster nodes before upgrade

touch /data/config/base/ospf_no_gr

  1. Perform software upgrade normally one node at time