In physics and engineering, you traditionally talk about things like "delta functions", write expressions involving them as if they are actual functions, etc. This is notationally very convenient but may be misleading because these things are not really functions.

So, what are they really? Well, the key things you can do with them are (1) "boring" linear algebra operations (you can add and subtract them, and multiply them by scalars) and (2) multiplying by some function and taking the integral. E.g., what delta(x) -- the Dirac delta function -- really is, is a thing such that when you compute integral f(x) delta(x) dx, you get f(0).

And so pure mathematicians have ways of dealing with them that make this property more explicit. The theory of distributions says: no, these aren't functions, they're linear functionals on the space of functions (e.g., the delta function is the thing that maps f to f(0)). So now you're no longer allowed to write them as integrals, which means that the very close analogy between "distributions" and ordinary functions is obscured, and e.g. if you need to do a change of variables you can no longer just do it the same way you already know about from doing integrals.

Alternatively, the theory of signed measures says: no, these aren't functions, they're kinda like probability distributions except that the total "weight" doesn't need to be 1 and the density can be negative in places. They are naturally applied not to points but to sets of points. (E.g., the delta function is the signed measure that gives a measure of 1 to any set including 0 and a measure of 0 to any other set.) Now you are allowed to write those integrals, but instead of writing integral f(x) delta(x) dx you need to write integral f(x) dH(x) where H(x) is the "Heaviside step function", so instead of delta(x) appearing there you have (morally) its integral, and again if you want to change variables or something you need to know a new set of rules for what you do to the measure.

Note: I have skated over some technicalities. They are quite important technicalities. Sorry about that.

The sloppy non-rigorous physicists' and engineers' notation, where you just pretend the damn thing is a function and manipulate it as you would any other function, is more convenient. (Right up to the point where you do some manipulation that is safe for actual functions but gives nonsense when applied to singular things like delta functions, and get the wrong answer.)

It's a little like calculus notation. The "Leibniz" notation we all use these days writes derivatives as dy/dx as if dx and dy were just small numbers (compare: we write integrals against distributions as integral f(x) delta(x) dx as if delta were just a function), which is kinda nonsensical if you take it too seriously but very convenient because it makes things like dz/dy dy/dx = dz/dx "obvious", which is not just coincidence but has something to do with the fact that derivatives really are kinda like quotients (in fact, they are limits of quotients). Similarly, using "function" notation for distributions lets you write things like "integral f(x) delta(x-3) dx" and see that "of course" that's f(3), and this convenience isn't mere coincidence but has something to do with the fact that distributions really are kinda like functions (and in fact every distribution "is" a limit of functions).

Newton had a different notation for derivatives. It didn't have a conceptual error baked into it (pretending that derivatives just are quotients), but it turns out that that's a useful conceptual error and that's part of why everyone uses Leibniz's notation these days.