So basically multiple CLS tokens. Fwiw, I tried multiple global tokens in my che... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Scene_Cast2 9 months ago \| parent \| context \| favorite \| on: Vision Transformers Need Registers So basically multiple CLS tokens. Fwiw, I tried multiple global tokens in my chess neural net and didn't see any uplift compared to my baseline of just having one.

ninja3925 9 months ago [–]

Note that it's not done for performance reason but rather to generate clear feature maps.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact