On 21 January 2026, an issue occurred in the Document Management capability on the NAR1b environment where document uploads appeared to complete successfully but later did not display in the user interface. The underlying documents were not lost. The issue was caused by a temporary failure in a background processing component responsible for applying document updates, so they appear correctly in the application.

Unplanned service degradation

· 21 Jan 09:30 – Background processing component responsible for updating document state stopped processing updates

· 21 Jan 21:06 – Root cause identified

· 21 Jan 21:29 – Service recovered and processing resumed

· 21 Jan 22:00 – Customer confirmed resolution

· Post recovery – Backlog processed and document state fully restored

· Document Management

· Impact was limited to tenants hosted on the NAR1b environment

· Document uploads could appear to succeed, but the resulting document state did not consistently persist in the user interface until recovery

· No data loss occurred. Once processing resumed, documents and related statuses displayed correctly

The incident was triggered by a planned infrastructure activity that overlapped with an automated restart of a background processing component.

During the restart, the component was temporarily unable to connect to a dependent backend service. A gap in startup error handling meant the component reported as healthy, but it did not actually begin processing updates. Because monitoring did not detect this “running but not processing” state, the condition persisted until manual intervention.

· The background processing component was manually restarted

· On restart, it successfully reconnected to required dependencies and processed the accumulated backlog

· Any downstream queues impacted as a secondary effect were drained in a controlled manner

The following improvements have been implemented to reduce the likelihood of recurrence:

· Strengthened startup logic with retries and controlled backoff when dependencies are temporarily unavailable

· Updated service behaviour to fail fast if processing cannot be initialised, enabling automatic restart and recovery

· Added monitoring and alerting that detects when processing is not active (not just whether the service is running)

· Added additional diagnostics to distinguish between idle and unhealthy processing states

This incident highlighted a resilience gap during infrastructure restarts where a background processing component could appear healthy while not actively processing updates. The changes above improve robustness, observability, and automated recovery, significantly reducing the likelihood of a repeat of this issue.

Root cause was a stalled asynchronous background processor responsible for applying document actions into the application state, which delayed event processing. The underlying infrastructure issue was resolved by the on-call team, the processor caught up within a couple of minutes, and documents and statuses then displayed correctly in the application.

Full RCA will be published in next few days

01/21/26 – 5:10 PM EST

Fenergo has confirmed that the issue has been fully resolved and all uploaded files are present and accounted for.

A preliminary Root Cause Analysis (RCA) will be shared within the next 24 hours, with a full RCA to follow in the coming days.

01/21/26 4.50pm EST

Issue appears to be resolved. Fenergo are continuing to monitor to ensure.

01/21/26 3:35pm EST

Fenergo are currently investigating an issue affecting document uploads on the Entity Profile Page and within Journeys. Uploaded documents are not persisting, and users are unable to waive or defer items in the Docs task.
Our engineering team is engaged and actively investigating.
​
Next update will be provided at 5:00pm