I don't have Azure experience but AWS Lambda is overcomplicated for REST APIs IMO. One reason is each function now has a deployment instead of just one deployment for the application. Frameworks can simplify it, but they don't eliminate all the moving parts. Lambda is great glue for filling in gaps in AWS itself.

Serverless can also mean something like EKS with Fargate. You get to use Kubernetes without managing any servers. Azure AKS has something similar with virtual nodes as I understand, though I haven't used them. I do think this model is better for long running services than serverless functions.