SSC 8.8 sseapi_rpc_queue: could not connect to SSE server

I recently upgraded my LCM deployed SSC server to 8.8. If you need a guide to go through the upgrade you can find my other post here.

After the upgrade was completed i was noticing strange behavior in the SSC UI so i checked the status the of the services. Here are the errors i found and how i fixed them

The first step was to check the status of the service

systemctl status salt-master

The return was this

root@ssc-01a [ ~ ]# systemctl status salt-master
* salt-master.service - The Salt Master Server
   Loaded: loaded (/lib/systemd/system/salt-master.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2022-05-15 16:39:26 UTC; 8min ago
     Docs: man:salt-master(1)
           file:///usr/share/doc/salt/html/contents.html
           https://docs.saltproject.io/en/latest/contents.html
 Main PID: 3035 (salt-master)
    Tasks: 39 (limit: 9830)
   Memory: 357.0M
   CGroup: /system.slice/salt-master.service
           |-3035 /bin/python3 /usr/bin/salt-master
           |-3041 /bin/python3 /usr/bin/salt-master
           |-3110 /bin/python3 /usr/bin/salt-master
           |-3115 /bin/python3 /usr/bin/salt-master
           |-3119 /bin/python3 /usr/bin/salt-master
           |-3122 /bin/python3 /usr/bin/salt-master
           |-3123 /bin/python3 /usr/bin/salt-master
           |-3124 /bin/python3 /usr/bin/salt-master
           |-3125 /bin/python3 /usr/bin/salt-master
           |-3203 /bin/python3 /usr/bin/salt-master
           |-3204 /bin/python3 /usr/bin/salt-master
           |-3206 /bin/python3 /usr/bin/salt-master
           |-3214 /bin/python3 /usr/bin/salt-master
           |-3216 /bin/python3 /usr/bin/salt-master
           |-3219 /bin/python3 /usr/bin/salt-master
           |-3220 /bin/python3 /usr/bin/salt-master
           |-3221 /bin/python3 /usr/bin/salt-master
           `-4871 /bin/python3 /usr/bin/salt-master

May 15 16:39:26 ssc-01a.corp.local systemd[1]: Started The Salt Master Server.
May 15 16:39:27 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
May 15 16:39:27 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused
May 15 16:39:27 ssc-01a.corp.local salt-master[3035]: [ERROR   ] Failed to get the salt environments: [Errno 111] Connection refused
May 15 16:39:28 ssc-01a.corp.local salt-master[3035]: [ERROR   ] Failed to retrieve commands from SSE: [Errno 111] Connection refused
May 15 16:39:32 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
May 15 16:39:32 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused
May 15 16:39:37 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_rpc_queue: could not connect to SSE server: [Errno 111] Connection refused
May 15 16:39:38 ssc-01a.corp.local salt-master[3035]: [ERROR   ] Failed to retrieve commands from SSE: [Errno 111] Connection refused
May 15 16:39:38 ssc-01a.corp.local salt-master[3035]: [ERROR   ] sseapi_event_queue: could not connect to SSE server: [Errno 111] Connection refused

The way i got around the error was by editing the /etc/salt/master.d/raas.conf. The file seems to be missing a key auth parameter. The engines section should look like this

engines:
  - sseapi: {}
  - eventqueue: {}
  - rpcqueue: {}
  - jobcompletion: {}
  - keyauth: {}

After restarting the salt master i was able to verify that the error was gone. To restart the service i ran

systemctl restart salt-master

To verify the status i ran

systemctl restart salt-master

* salt-master.service - The Salt Master Server
   Loaded: loaded (/lib/systemd/system/salt-master.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2022-05-15 20:02:56 UTC; 51s ago
     Docs: man:salt-master(1)
           file:///usr/share/doc/salt/html/contents.html
           https://docs.saltproject.io/en/latest/contents.html
 Main PID: 31309 (salt-master)
    Tasks: 39 (limit: 9830)
   Memory: 330.0M
   CGroup: /system.slice/salt-master.service
           |-31309 /bin/python3 /usr/bin/salt-master
           |-31315 /bin/python3 /usr/bin/salt-master
           |-31320 /bin/python3 /usr/bin/salt-master
           |-31323 /bin/python3 /usr/bin/salt-master
           |-31325 /bin/python3 /usr/bin/salt-master
           |-31326 /bin/python3 /usr/bin/salt-master
           |-31327 /bin/python3 /usr/bin/salt-master
           |-31328 /bin/python3 /usr/bin/salt-master
           |-31330 /bin/python3 /usr/bin/salt-master
           |-31397 /bin/python3 /usr/bin/salt-master
           |-31398 /bin/python3 /usr/bin/salt-master
           |-31400 /bin/python3 /usr/bin/salt-master
           |-31411 /bin/python3 /usr/bin/salt-master
           |-31412 /bin/python3 /usr/bin/salt-master
           |-31413 /bin/python3 /usr/bin/salt-master
           |-31414 /bin/python3 /usr/bin/salt-master
           |-31415 /bin/python3 /usr/bin/salt-master
           `-31416 /bin/python3 /usr/bin/salt-master

May 15 20:02:54 ssc-01a.corp.local systemd[1]: Starting The Salt Master Server...
May 15 20:02:56 ssc-01a.corp.local systemd[1]: Started The Salt Master Server.

VMware documentation also talks about the procedure above in the Upgrade the Master Plugin documentation found here

If the status page a return similar to this

 [py.warnings      :110 ][WARNING ][5488] /usr/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.11) or chardet (4.0.0) doesn't match a supported version!
  RequestsDependencyWarning)

Follow my other guide here