Thread Safety in Azure Logic Apps

Development

After resolving the previous issue with the Content-Type defect in the Azure Logic App’s storage connector, I ran into a second problem: The Web pages being generated were populated with the correct content type, but the incorrect content. I quickly discovered that was partly caused by a race condition and with how memory is managed by Azure.

By default, a Logic App will run up to twenty instances of a ‘For Each’ loop in parallel, but this is multi-threading instead of true parallel processing.

A variable used by a ‘For Each’ loop must be initialised outside of it. The variable becomes a shared buffer for all iterations of the loop, and therefore for all the threads running in parallel.

What I think happened, in this case, was each thread would read from the variable before it finished overwriting the data from a previous thread. This is the race condition.

What to do? Two options came to mind:

  1. Enable concurrency control and limit to single threading. This is unworkable when there are a large number of iterations on large amounts of data, and I discovered it doesn’t actually resolve the problem.
  2. Another option I considered was to have the variable store a JSON array of objects, and to have each iteration of the loop append to that array before reading off the last object. This would be a complicated solution, and I know from experience that Logic Apps don’t really work with large JSON arrays.

Then I remembered having used something called ‘Compose’ in another project. The Compose step is intended primarily for JSON arrays, I think, so it can be used to store any string (or recast and store variables as strings, in my case) without the need to initialise it first. Using it in a loop should result in a buffer being created for each iteration. The risk is the buffer might not be disposed after a thread completes. Still, it was the least worst option. And that solved the immediate problem. The Logic App generated the pages, with the correct data, and it didn’t exhaust the memory to the point of slowing the execution of the Logic App.