Vscode was always going to attract such issues. On my system, the app does not have access to the home directory and everything is done on a remote container that I locally ssh into (thanks to flatpak's bubblewrap and docker). As a result everything is cleaner and vscode is isolated from the host.

Access to local folders on the host (though rare) is approved on a needs basis.