1 problem

0 votes0 replies0 views
The attention-head information conjecture for salient network structure
The attention heads in a transformer network carry information about relationships among input tokens. Attention-head information conjecture. Exploiting information carried by the…
Conjectureopenattention-mechanisms network-interpretability transformersarXiv posted 2 days ago

The attention-head information conjecture for salient network structure