RCA
Incident Summary
Incident Date | 18 June 2026 |
Type of Event | Unplanned |
Severity | Sev1 |
Service Affected | Journey Task Processing |
Impact Start | 18 June 2026 — 3:16pm |
Service Restored | 18 June 2026 — 4:21pm |
Duration of Impact | Approximately 1 hour |
Status | Resolved |
Detailed Event Narrative
On 18 June 2026, following a large load event, certain tasks within client journeys temporarily slowed progression across a number of production environments.
The investigation identified the root cause and the source of the additional processing was identified and stopped. Normal task processing resumed and all affected journeys progressed without any manual intervention or loss of data.
Event Timeline
Time | Event |
18 June 2026 — 3:16pm | Large load event triggered. |
18 June 2026 — 3:20pm | Monitoring alerts triggered. Client Journey Processing (Task processing) slowed. Engineering investigation commenced. |
18 June 2026 — 4:20pm | Recovery, incident resolved |
|
|
Services Impacted
The following services were impacted during this incident:
Service | Impact Description |
Client Journey Task Processing | Certain tasks within client journeys temporarily delayed progressing across a number of production environments. |
There was no impact on the accuracy or integrity of work completed prior to the incident. No data loss occurred.
Affected Applications and Systems
The impact was contained to the client journey processing (tasks) area of the platform; all other areas of the Fenergo application continued to operate normally throughout the incident.
Recovery was completed across all impacted environments on the same day.
Root Cause Analysis
The root cause of this incident was an unusually high volume of processing activity temporarily consumed shared system capacity that is also used for standard task processing.
The activity triggered a much larger volume of downstream system processing than is typical. This additional processing temporarily used up shared system resources, which in turn slowed the normal processing that drives task progression for other journeys.
An existing system behaviour meant that certain bulk actions could generate a larger amount of downstream processing than expected. Work to address this behaviour was already underway prior to this incident.
The final corrective deployment resolved the issue at its source. All affected tasks were subsequently reprocessed, and impacted client journeys were unblocked on the same day.
Corrective Actions and Corrective Actions Timeline
Fenergo has identified the following corrective actions to prevent recurrence:
# | Corrective Action | Target Timeline |
1 | Introduce additional limits to prevent bulk actions from generating disproportionate downstream processing load. | Done |
2 | Continue migrating relevant configurations to an improved processing model that does not exhibit this behaviour. | In progress |
|
|
|
Closing Statement
Fenergo takes the reliability of its platform and the continuity of our clients' operations seriously. We are committed to delivering the corrective actions outlined above and to embedding the lessons learned from this incident into our engineering and deployment practices.
We apologise for the disruption caused and thank our clients for their patience and continued partnership with our support teams throughout the recovery process.
18 June 2026 17:00 GMT
System has recovered since 16.30. Full RCA will be published in coming days
18 June 2026 16:30 GMT
Fenergo are monitoring and observing system is recovered. We will continue to monitor and will provide RCA in coming days. All journeys will processed including those affected
18 June 2026 16:25 GMT
Fenergo are monitoring and observing system is recovering
18 June 2026 16:05 GMT
Fenergo monitoring has alerted us to an issue with the Tenants on EMEA1
This is resulting in service degradation and journeys becoming stuck on tenants within that region