Incident report 14/09/2023
Start: Toulon/1.7.2 - 8:00pm
End: Toulon/1.7.3 - 11:45am
Reach: limited to some playbooks (see below)
Impact: high - workaround
Description
Section titled “Description”During the rollout of Toulon release, we encountered a bug affecting certain playbooks. This issue arose when a variable is assigned the value derived from a preceding variable within the same Transform data step.
This document shortly details the cause, the resolution, and the actions that will be undertaken. The Mindflow team apologizes for the inconvenience. We also thank our clients’ teams for their support and understanding.
Causes
Section titled “Causes”- Upon receiving the report, the Mindflow team promptly replicated the issue and identified a regression in the workflow engine’s expression evaluation algorithm. This anomaly stemmed from a variable failing to inherit the value assigned to a preceding variable during the Transform data step, which consequently led to null values, thus causing certain playbooks to fail.
Workaround
Section titled “Workaround”- Using hardcoded values in the Transform data emerged as a viable workaround. Subsequently, we found that segregating the variable assignments into two separate Transform data steps also effectively resolved the issue.
Resolution
Section titled “Resolution”- The Mindflow engineering team has fixed the algorithm, solving the problem affecting some playbooks by restoring the proper processing of connected variables.
Follow-up
Section titled “Follow-up”- The Mindflow development team will continue to enhance its test suite to better address workflow engine variable evaluation issues.