Back prop requires and limits to analytically differentiable in a normal way.

Attention is... Oh dear comparing linear regression to attention is comparing a diesel jet engine to a horse.