libobs: Fix wrong order for source->async_mutex

[walker-WSH]

Description

same with #11687
For basic logic in libobs, gs should be requested before async_mutex.
In async_tick, there is no mutex locked before, so here the logic is being async_mutex->gs.
If another thread happens to be requesting gs->async_mutex, nested locks may cause UI block

Motivation and Context

keep stable order for requesting mutex.

How Has This Been Tested?

add media source, try to switch media file with different resolution, ensure video is display normally

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[exeldro]

Would it not be better to move set_async_texture_size outside of the async mutex like it is in deinterlace_update_async_video ?

[walker-WSH]

hi @exeldro I checked deinterlace_update_async_video and find it has multi-threaded safety issue.

this functions is called from graphic thread, and it is accessing prev_async_frame outside of async_mutex.
However, prev_async_frame must be accessed in async mutex because its lifetime is managed inside this mutex.

it may be free from other thread, such as callback of media source.

from graphic thread

eg: from decode thread of media source