More people were complaining about such issue, like here:
or here:
Second one was already dead, but still produced it’s noise. I don’t know if it came back to radxa and was diagnosed or not. Ask @setq for details.
Try to power HAT without board, check if noise is still coming from it,
Do You have any thermal pad to cover element near power jack? You may try to find if this one cause issue.
You may try to replace Your HAT, obviously not everyone has this issue,
Maybe problem will not be noticeable under some heat pad, heatsink and in case, original one is really thick and probably will solve the problem.