I am wondering which languages have this "reliable by default" property. For example, even Mathematica stores floating-point numbers as double-precision.

Mathematica (default settings):

  In := 0.2 - 0.10000001
  Out = 0.1

Out of curiosity, why is this property specifically desirable for deep learning applications? A neural net typically uses floats, and I suspect it would be robust to very minor error.