Sort order for delete makes a difference

[hazvk]

I'm currently using repo bootstrapped from https://github.com/Azure/AzOps-Accelerator

When tinkering with the setting "AzOps.Core.CustomTemplateResourceDeletion": true, I had a couple of findings:

Expectation:

• Resources should be deleted in an order we specify AZOPS_CUSTOM_SORT_ORDER to "true"
• Furthermore, given we deploy resources in a certain order, the resources should be deleted in reverse order.

Actual:

• I found that deletion wasn't being covered in an ordered way, even after setting AZOPS_CUSTOM_SORT_ORDER to "true"
• Resources were trying to delete in the same order we created them, which led to dependency conflicts on delete

Side note: I also found a scenario of failure - if we delete files from the state, and modify files out of the state, the validation and deployment will fail.

My suggestion is to align to above expectation. Keen to hear thoughts.

[hazvk]

By the way - this idea flows on from an existing idea #876.

I have already made custom implementations to the template to make sure that AZOPS_CUSTOM_SORT_ORDER is considered on Validate and Push. It worked quite well for add/modify, but this is a branched discussion about how it can be improved for deletes.

[Jefajers]

Hi @hazvk, great consideration I will take a more detailed look at your accelerator thoughts and especially with reverse order in the upcoming days.

Could you elaborate on this a bit I am not sure I fully understand the scenario:

Side note: I also found a scenario of failure - if we delete files from the state, and modify files out of the state, the validation and deployment will fail.

[hazvk]

Thanks @Jefajers looking forward to an update!

In terms of this scenario - this was a bit lazy of me to bundle it into one issue...

Scenario

I am trying to use the custom template deletion, so I have made changes in settings.json as well as deleted some templates in root/

Steps to Reproduce

1. In an existing bootstrapped repo with formed state inside folder root/, delete some templates that have already been created
2. Make at least one change in file(s) outside of root/ e.g. within .github/, settings.json, .scripts/
3. Raise PR to trigger Validate workflow

Results

Expected: validation completes and results are printed
Actual: error occurs, as there are no valid templates to add/modify resources

[Invoke-AzOpsPush] Expecting deploymentList object, it was not found. Error exiting runtime.
Exception: ScriptHalted
Error: Process completed with exit code 1.

Screenshot:

[hazvk]

Suggested Solution (to above scenario)

When I saw this, my instinct was "we shouldn't consider diffs outside of the defined state when ascertaining diffs for the validation and push"

I've snuck in a change as part of the larger PR behind this discussion, it's pretty much captured via Github Actions and Pipelines changes respectively (3 lines each):

• https://github.com/Azure/AzOps-Accelerator/pull/146/files#diff-ce8741d9ef7e55f7f273e760f59bbca5c2fef322e458db024cfc4be7d39dcf68R22-R24
• https://github.com/Azure/AzOps-Accelerator/pull/146/files#diff-0fad57c9e0b4a60556b0635904595f3fe9c6e4931fc4c68c637de34954cb73daR18-R20

Screenshot of logs after success

Note: no error and diff used to ascertain deployment only involves files in state. Instead of last line being an error, it moves to start removing resources.