Asynchronous workflow execution
Unlike mistral actions, Mistral currently doesn't support asynchronous workflow execution.
I think it would be beneficial to have asynchronous workflows for these reasons:
1) Workflows carry context, and depending on the size of the workflow and the contents of the workflow variables, this could become a very heavy context. It would be nice to keep these contexts separated if they don't share any dependencies.
2) If a workflow is responsible for executing hundreds (or thousands) of other workflows, I want to be able to fire and forget and check back on those workflows individually. If the state of the parent workflow is still necessary, then you can let those child workflows complete and then update the state (similar to how asynchronous actions work)
3) The API will block because it waits for a response from the engine client, so when there are many workflow execution API calls coming in together, it will overload the API.
rakhmerov's notes:
#1 is not clear to me. What contexts need to be separated?
#2 is not a concern, "workflow <-> subworkflow" communication is asynchronous in the first place.
#3 is a concern, yes, and this is something we can deal with by adding a workflow parameter that enables asynchronous workflow execution creation. E.g. params={"async": true}. In this case we can just return ID of a workflow execution immediately for further tracking and don't wait till the workflow execution is fully created, validated etc.
Blueprint information
- Status:
- Started
- Approver:
- Renat Akhmerov
- Priority:
- Medium
- Drafter:
- Renato Recio
- Direction:
- Approved
- Assignee:
- Renat Akhmerov
- Definition:
- New
- Series goal:
- Accepted for ussuri
- Implementation:
- Started
- Milestone target:
- None
- Started by
- Renat Akhmerov
- Completed by