Is that enough? If I put caddy in front of my existing infrastructure am I now secure and doing TLS correctly? That's what I did but I have no idea if some expert wouldn't look and say "no, this is not done correctly, here's all the parts you missed"
Probably not. The default ciphers are reasonable. HSTS isn't on by default, last I checked. But Caddy does generally seem to aspire to do the right thing by default.
There is a Mozilla tool called Observatory that seems pretty comprehensive in checking TLS setup and some other security settings, headers, etc: https://observatory.mozilla.org
Yes, it's enough for most people. Companies have thrown Caddy in front of their infrastructure last minute before losing PCI compliance due to TLS management problems. As others have said, Caddy doesn't do security headers by default (yet?), but its default TLS parameters are very good.