Wow I did not know that, I was always under the assumption that I needed to have a project using the Microsoft.NET.Sdk.Web for anything Web as its base. Thanks for the info!
It used to be much more modular back in dotnet core 2.x. It was just too complex for most people to wire up everything themselves. You needed to install a lot of nuget packages, add a lot of middlewares. In the end 95% of the projects added everything anyway, but always with some little mistakes and weird errors.
Starting with 3.0 (or 5.0?) they ditched the Startup class and just added in everything by default. Much easier for the regular web application. The modular approach is still everywhere though. You can just pick the components you need, most of them also run without DI, it's just a bit of a hassle to manage all those dependencies manually.
FrameworkReference enables other cursed combinations, too. You can use WinForms/WPF in an ASP.NET project with a FrameworkReference to Microsoft.WindowsDesktop.App. Finding a use case for this is left as an exercise for the reader.