What if that point is 20 years from now? What if every time Ford/GM/Toyota substantially changes the look of their cars, your classifier no longer recognizes them because all your data only has the old models in it. That's what people are driving at. Simply collecting more data is not enough to solve this problem.
At a certain point it's just about recognising an object which shares broad characteristics with a car rather than aesthetics. Eg it moves at the speed a car moves at, it's in the road, it's overtaking on the right hand lane. I would expect any autonomous car to be able to fail over to "this object is likely a vehicle I haven't seen before" given a strange car-like object being detected.
Great. Now the problem you've posed is no longer image classification. It's more like video classification or zero-shot classification! (neither of which are close to solved)
It doesn't seem like zero-shot classification to me. It still seems like image classification. You said:
> What if every time Ford/GM/Toyota substantially changes the look of their cars, your classifier no longer recognizes them
My answer was probably incomplete, but I took the above to mean that cosmetic changes to vehicles mean that classifiers no longer identify them as cars, and this detrimentally modifies the behaviour of the car.
Whilst it's trivial to envisage a scenario where your problem is solved systemically (sufficient training data for a new chassis released in advance or something), it seems like it would be possible to train based on "things we expect to see from any car".
As far as I know, that's how all of the existing methods operate. They seem to have a hierarchy for decision-making:
0. Is there an object around me which I need to consider? If not, continue to monitor for one whilst operating the vehicle within the parameters of road signs and conditions.
1. Is this an object which has a predictable path based on either its signals or the expected behaviour of a car on this part of the road / operating within the parameters set by the road signs I can see?
2. Is this an object which is operating safely despite not falling into category 1?
3. Is this an object which I need to take action to avoid?
Which is to say that it ought to be possible to "fool" a Tesla with a non-car object behaving in a similar fashion to a car. The Tesla sees an object, not a car.
"it moves at the speed a car moves at, it's in the road, it's overtaking on the right hand lane" is video classification, which is not solved. In fact, at least how you described it (you could probably change the problem statement to avoid this), this would involve an ML model that must learn a model of physics - also unsolved.
You've just specified a manually hardcoded set of decision rules. This is not machine learning, and is incredibly brittle.
I had thought that in your original post you were agnostic about the methodology for identifying a car, but were remarking that, in a world where it's possible to do it using whatever form of classification, it would be possible to 'stump' any reliable model by modifying the appearance of a car. I'm observing that any model for classification almost certainly would not rely on aesthetics.
> You've just specified a manually hardcoded set of decision rules. This is not machine learning, and is incredibly brittle.
I'm pointing this out to illustrate that the technology already deployed to solve this problem does not get confused by aesthetics.
I was talking about deep learning. The comment I was replying to was making the specific problem seem as if it were easy. Certainly there may one day be a classification technique that does what you say will do. But you may as well have said there will one day be a perfect classification technique that will just perfectly output steering angles, end thread. What use is there in conjecturing about perfect unknown classification techniques? Not to mention that there is no guarantee such a perfect method would not rely on aesthetics. Even if the train set has more than just aesthetics (e.g. video of cars in motion) maybe this perfect classifier would just cheat and rely on aesthetics, you don't know.
So I'm pointing out the methodology you suggested is not currently feasible, or is currently widely considered by the community to be the wrong practical approach. Because theoretical solutions will not solve self driving cars.
So you put in your training and test dataset a bunch of such situations. At some point you've covered enough cases to extrapolate the rest.
Good testing is going to hunt for these blind spots and fix them. Fact is that it's already safer than humans, even with all its hidden imperfections.