No, it's not a transverse wave in the sense that you're describing, like a wave in water. It's a wave in the E&M fields, both of which are 'vector fields'. Mathematically, we describe both electric and magnetic fields as a vector (a direction + length, like a little arrow) at each point in space. Light waves are oscillations in the lengths+directions of these vectors. There's no medium moving transversely, but the vector field points in the direction and with the magnitude drawn in those squiggly line diagrams.
See Maxwell's equations, which describe how those vector fields change over time. They do not describe how any sort of medium moves over time.
Yeah, you're right. I don't mean transverse in that it is a wave in water. There is no 'water' particles there to be moving up/down left/right.
Interestingly, Maxwell himself worked in a time when the aether model was dominant; that there had to be something for the waves to be moving in/through. His equations held up under both the aether model and special relativity which made aether unnecessary.
See Maxwell's equations, which describe how those vector fields change over time. They do not describe how any sort of medium moves over time.