"Especially when your opinion goes against what pretty much everyone else in the field thinks" -> Most Deep learning people are consumers or hardware manufacturers, very few researchers in the field, and most people that have deep knowledge of it would agree with this guy.
I believe he has much more insight than most AI researchers/engineers. What's going on in AI right know looks exactly like what happened in the 1960s, sucking all the resources into something (i.e Deep Learning) without looking at the big picture, without looking at epistemological questions, and not training new grads on solid AI foundations anymore like probability theory and logic. Not asking the right questions in terms of expressivity of models, sensitivity analysis, robustness and manifold learning. The next AI winter is coming.
Yes, but the next AI Winter will be far less bleak and job-free as that of the 80's, IMO. The hayday of DL-based AI is likely to continue for 10+ Years, as we refine the methods and see how far word2vec and improved compositionality of the many existing DL methods takes us. AI is likely to improve rapidly for another decade, IMHO. But I agree that DL AI, if not augmented substantially, will stop well short of AGI and consciousness.
But 10 years is a long time in internet years. I would not be at all surprised if there's another word2vec (or two) arose before then to displace the tectonic plates yet again.
Indeed, DL-based AI is likely to continuue - rightfully so - for non-safety critical domains (translation, games, ...) It's crazy that few people notice that real world applications with safety issues (where 99.9% accuracy is a joke) - ex: self driving vehicles - require much more that what DL has to offer. The AI community's sloppiness when it comes to safety and robustness engineering will lead to the some serious problems (when it comes to real-world industrial applications). But yeah there will still be jobs for people playing around.
Obviously, "Lane Following Autopilot using my brain and controls theory" would not make it to the top of HN. Welcome to the new era where Tensorflow replaces Lyapunov and ML spares you the need of understanding hard problems... until you need guarantees and safety... but but it's ok let's add more data.
I agree with you. If you can leverage control theory from the 1950s to solve your problem, what's the point?
However, I will state that using e.g. Lyapunov functions to prove the stability of the system requires a model of the system. And even if you need a guarantee for your system, that guarantee is only as good as the fidelity of your model. For an inexpensive RC car, with slippage and saturation, without torque control or inertial sensing, you're going to have a hard time doing something that sounds as principled as what you suggest.
You seem to be forgetting the entire vision pipeline that automatically extracts "lanes" and that information gets incorporated in an end to end manner requiring only true steering angles and nothing else. Its easy to comment but its not as straightforward or trivial as one might assume.
Indeed, I was not really talking about the vision pipeline. But once you decouple the problem (use ML for vision, planning for the trajectory, controls for the rest), you'll get much more stability, guarantees and insight into how to improve your problem. These kinds of end-to-end approaches are very hard to evaluate, they have zero educational value, are not parsimonious and tend to reduce people's analytical skills.
Tedious, and also solved for a decade already. Also, it's much easier to just find lanes using traditional CV and simply using annotators to verify the lane labels.
You can never use 1950s control theory to solve your problem? I think you didn't understand the my comment, so please let me clarify: I was claiming that even the control problems in this RC-car-lane-keeping domain can benefit from learning approaches.
Rather than demeaning someone's effort, learn why specific methods are considered appropriate/state-of-the-art when solving certain problems.
This is an unbelievably wrong comment. All the Lyapunov and traditional Process control theory in the world won't help you solve autonomous driving.
Also regarding "Guarantees and Safety" they don't magically appear out of thin air when you use traditional process control especially in noisy domains like autonomous driving.
This comment is equivalent of "I can write code to solve Atari Pong in any programming language deterministicly so any post showing Deep Reinforcement Learning is stupid"...
So control theory is ok for unmanned aerial drones but autonomous driving is just too far? Control theory can't handle noisy domains?
Guarantees of safety (more accurately stability) is the entire point of lyaponov analysis, and it's used on noisy systems all of the time (https://www.mathematik.hu-berlin.de/~imkeller/research/paper...). Can you point to a specific noisy system that control theory is ill suited for?
Once you have a path to follow, classical control theory can be used to control the steering angle to follow it.
But classical control theory hasn't been able to extract, from camera pixels, the open path in a road with cars, bicycles, and pedestrians. Camera inputs are million-dimensional, and there aren't accurate theoretical models for them.
Unmanned drones are orders of magnitude easier since you don't have anything that you can just fly into once you are above few hundred feets. They also don't have to rely on any vision based sensing. E.g. a drone has altitude, current speed, heading all of which while noisy can be represented easily as a small set of values.
The whole Lyapunov and control theory assumes perfect knowledge of sensors. Even though the signal itself might be error prone you have a signal. In case of autonomous driving even in simple cases as those described in the blogposts knowing the exact position of the markers and then using them to tune the contoller is not as easy as you might think.
The end-to-end system shown here solves three problems it processes the images to derive the signal, it then represents it optimally to the controller and then tunes the controller using provided training labels.
I cited Lyapunov, more as the ABC of nonlinear controls. Much more can be done in an analytical fashion, the "end-to-end" system here does not "solve" anything. It is a trained steering command regressor, nothing fancy, it's likely to work in this guy's living room, under certain lighting conditions, there is no way of predicting its accuracy, sensibility or anything else. Engineers have been breaking down systems into sub systems for a reason -> tractability of testing and improvement. End-to-end systems like that have close to zero value if you need something reliable.
Obviously, my message was slightly provocative, deep learning methods and classical controls (which by the way are able to quantify robustness to plant uncertainties and noisy signals) are all very useful but shall be used in combination. End-to-end techniques that bundle perception, planning and control in an opaque net are fun to play with (like in this article), it just very sad to see people believing this produces robust and safety-critical systems and we see too much of such articles on HN.
I agree with you in the sense that if a known and reliable way to map knowledge and information from one domain to another (e.g. from desired trajectory + perceived current position to steering inputs), I'd much prefer that than black box ish neural nets. Neural nets aren't meant to be the silver bullet.
But in this case though, any kind of state space control also requires rather precise knowledge of the physical laws that govern the dynamics of the vehicles. When such information is not available, can neural nets do a decent job at mimicking an analytical control algorithm? I think that's an interesting problem worth exploring.
When falling down actually corresponds to killing a pedestrian, then I'd rather try to understand the complexity of robust walking rather than observing a bunch of humans, mimic their behavior and hope for the best.
Also, general autonomous driving isn't as hard as it sounds. Given a list of time-dependent coordinates of obstacles, it should be pretty easy to navigate around such that no collision occurs. The hardest part is testing, but this is just a matter of tedious work and doesn't require great intellectual effort.
> Given a list of time-dependent coordinates of obstacles
A tree falls in an intersection because some carpenter ants chewed through trunk. Cars swerve to miss the tree and collide in an inelastic ball of nonlinearity, showering debris everywhere. You approach this at 65 mph and have 23 ft to decide what to do. Fear not, you have a list, a perfect list with coordinates, velocities, and material properties of every solid body in the area. Furthermore, without great intellectual effort, you can solve the millions of coupled differential equations that govern the dynamics of the entire system in near real time. Oh, and your list also has a measure of importance of each bit of mass, whether it is human, animal, or inert. And your list also accounts for the degrees of freedom introduced by every other car approaching the intersection, also using their own respective lists and perfect knowledge of the world to miss each other?
I think Rockets are straightforward too just a bottle with expanding gasse through a series of nozzles, pointed at different angles at correct time but since I know I am not a rocket scientist I dont go around claiming moon-landing was not a "great intellectual" effort.
Completely agree with this viewpoint. Explanatory power and reproducibility are two of the most important aspects of a scientific paper. Most deep learning based papers lack both these properties.
> Explanatory power and reproducibility are two of the most important aspects of a scientific paper.
The reproducability problem should be solvable if the authors would simply release the training data and source code that trains the network (as yould scientific practise would require). Or is there anything else that is necessary to assure reproducability (I'm not that deep into CV papers)?
In aeronautics, we were able to increase safety to an insane level by understanding the physics of the environment, formally proving and certifying algorithms, using Robust control theories that allow to formally deal with uncertainty. The power of mathematical modeling together with robust software testing and a limited/controlled use of learning algorithms is - in my opinion - likelier to bring safety to such systems rather than such an opaque use of CNNs. And I'm not even talking about human machine interaction or liability issue that would emerge from such extreme approaches (which node of the net or which training sample will be blamed?). I guess CNNs allow a lot of wannabe engineers to play with real world problems and dream that their 10 lines python code would match semantic models if fed with more training data, but I'm pretty sure Aircraft/Car manufacturers will/should not replace formally certified controls algorithms and redundant architectures built on the top of hundreds years of analytical results with a rack of NVIDIA GPUs.
This is actually completely incorrect. Many cognitive biases occur at low levels of cognition without involving language, take professional pilots and confirmation biases despite of a ton of training data.
I think professional pilots are still making mistakes at a higher level. By "low level" I mean things your brain does in a split second without thinking about.
You're so right, actually there is nothing new in this idea. Kalman filters (so basically belief propagation based on gaussian observations) were already used by DL Kleinman in 1970 to model aircraft pilots behavior. We are way past this simplistic hypothesis. Actually, quantum dynamics (yeah why not, it's just a model) seem to better describe human decision making than classical stochastic models (Quantum dynamics of human decision-making - http://www.sciencedirect.com/science/article/pii/S0022249606...)
I feel like frameworks like Meteor promote systematic use of websockets, and more and more developers don't even try to implement scalable stateless server code.
