Looking at the code, the good stuff is in the binary blob. The binary blob returns polygons for the edges of the road, with a probability for each. Path planning, in "pathplanner.py", consists of a probability-weighted average to find the center of the lane, using an assumption of 3.6m lane width. There's no map-building at all. It's purely reactive.
Speed control uses data from a radar and from the vision system. The path info is used to decide which targets the radar should consider important. The big thing seems to be identifying the "car ahead", if any, and maintaining an appropriate distance.
There's a hard-coded 6 minute timeout to make the driver take control once in a while.
On a good day on a well-marked freeway, this might work. It would be interesting to modify this to process pre-recorded dashcam data and see how well it tracks the road.
Based on a really cursory inspection of the binary and symbols, the binary seems to be an OpenCV project which uses some common image processing algorithms (Sobel edge detection, Hough to find straight lines).
There doesn't seem to be a large amount of embedded data indicative of a trained model or neural network.
So, it seems that the vision algorithms probably mirror those in the common literature and represent a reimplementation or iterative improvement on existing lane-finding systems.
More reversing would probably yield greater insight :)
I think it's also using Qualcomm's FastCV, judging by the fcv* prefix on a lot of the symbols. Which makes sense, since FastCV is optimized for the Snapdragon processor in the OnePlus 3 phone they use.
visiond appears to be linked against libgsl.so. To my understanding, distributing a binary blob using the GNU Scientific Library is against the license of this library. That written there does not seem to be any symbol used from libgsl.so.
This seems to be a code dump from the comma AI startup. It also indicates their autonomous driving platform runs on a "stripped version" of Android.
Which seems to indicate that they are clinically insane, given they are using a system that can easily have excessive scheduling latencies under load to steer a car.
People like me laughed at Android platform adherents because audio latency was a huge hurdle to even remotely be on par with Apple's Core Audio, but, believe it or not, 8 years later (give or take) Android has caught up. Point being, what might be "clinically insane" to you is simply a "fun challenge" to people who may be smarter than the both of us combined.
It's not about being clever with Android programming; it's about the fact that Android is not by any stretch of the imagination a real-time system and should absolutely not be controlling a car.
This is great IMO. I was incredibly dismayed to hear they were canceling the Comma One without any further action, without even attempting to work with regulators. This seems like the culmination of that - now the responsibility for safety is in the hands of individual users/developers and not on Comma, which is what got them in trouble in the first place.
Now that this is open source though, I wonder what their next project will be?
I have my reservations about this code. I am a long time free software developer so it is hard to say it but it seems clear to me that a single entity (from a legal perspective) has to "own" the self-driving car software.
In the future bits of a self-driving car system may get open-sourced the way Facebook Open Compute has emerged in data centers, but first the proprietary implementations will need to pave the way in the legal world.
It's not all bad news, though: I'd love to see a free software project that converted my lawn mower into a self-driving lawn mower...
> it seems clear to me that a single entity (from a legal perspective) has to "own" the self-driving car software.
That's not clear to me at all. It seems clear to me that the person operating the vehicle has ultimate responsibility for how it is driven, whether they're personally driving it or not.
That's not clear to me at all. It seems clear to me that the person operating the vehicle has ultimate responsibility for how it is driven, whether they're personally driving it or not.
Seems reasonable at first glance, but responsibility makes only sense if you also have the knowledge and power needed for making the right descisions.
Unless the driver is expert in the self-driving car field and core contributor to the car software, he'd end up being liable for events he cannot anticipate and has no control over. In such a situation, the only actually responsible descision would be not to use the car at all.
You can solve that problem with certification. Sure, the software platform is open but actually building a car and getting it to the road requires certification of the final product.
I don't see why. Liability for other dangerous, complex artifacts don't necessarily follow that rule, even when it comes to software, and even when it comes to open source software.
GPSd is one example. Used pretty much everywhere, including life-and-death real-world situations. Consumer products (like your car's GPS) including it tend to pop up a "don't be an idiot, but we're not liable even if this malfunctions" screen. (The Pentagon and oceanic research operations, well, I'm the wrong person to ask about how they handle software liability, but kinda doubt we'll see class actions from either.) But the important part is that that screen is not there to protect the authors of GPSd. It is there to protect the VAR.
As in research projects, which this project expressly states that it is? What are those reasons? It seems like being able to test theories as quickly and cheaply as possible is the top priority. If Python, a general purpose OS, and a CAN-USB adapter is sufficient to conduct the research and satisfies those attributes, it seems like the perfectly logical way forward.
I understand why you would want to move to such systems you describe once your research has concluded and you are now building something for production, but that is not the intent of this project, at least at this stage. It specifically says so.
> As in research projects, which this project expressly states that it is?
This is from comma.ai, who famously tested on public roads with a reporter in the car and very little prior testing ("the first time it worked was this morning").[1]
Why wouldn't you do the research on a platform that could actually be deployed? Build the core using the proper tools and techniques. Any bug in a system like this could be life threatening at worst. Wouldn't you want to use the research phase to help eliminate such bugs?
Unless you're not planning on ever releasing then a platform like this makes sense for strictly research, but Comma AI did plan on releasing their product until they got shut down.
Research implies that you don't know the outcome and are going to be throwing away everything that doesn't pan out. Without knowing the outcome, how can you really be sure of what you need? You might think you need a CPU-based system but learn that GPUs are necessary, for instance. If you can start with cheap off-the-shelf components, at least you don't have much to loose when it is time to throw it away. If you're spinning your own boards, or heavily tied to expensive development kits, that becomes much harder to swallow. Especially for a startup with limited resources.
> Wouldn't you want to use the research phase to help eliminate such bugs?
I wouldn't think so. Spending your time fixing bugs in something you realize could have been done better another way, which ends up getting thrown out, seems like a waste of time. The words research and development are often paired together because it is a two step process, where development comes after you have learned what can be done.
As an example of some requirements, any recursion is banned unless accompanied by a termination proof that keeps track of stack size. No calls to malloc() or free() are allowed, and so any library you might use that allocates memory is banned.
It's far easier to get the code working in something like Python, and then rewrite it in safety-critical C later. And the kinds of people who can write self-driving cars in Python are using very different skills than the kinds who can write safety-critical C, so it might not even be the same person.
Hmm. Clearly this code is miles away from MISRA compliance (although compliance doesn't mean that the software is any good)
Even so, sorry to say, the code is miles away from any decent software standard. It is almost exactly what you might call CRAP (classes really a procedure) and mixes responsibilities all over the shop. Little attempt is made to abstract behaviour, and there are differing units, and random multipliers all over the place.
Using c++ and python, both of which are OO many classes of error could be avoided if any actual OO features were used, even if MISRA was not a target, sadly the code doesn't do that.
This has all the hallmarks of software that doesnt know if it's doing m/s or mph.
Exactly; the delta from Python to C is smaller than the delta from C to "safety-critical real-time C". Might as well prototype and simulate in a higher-level language that's obviously not the final one, rather than writing in a language someone might mistake for production and trying to incrementally evolve it into a full safety-critical version.
> once your research has concluded and you are now building something for production
I think the rub is that this code appears to be intended for "production" (or at least use on public roads), given the fact that it's published by a company that has tested its products on open roads?
It explicitly states: "THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."
The big problem in my mind is that Python is the wrong tool for the job. It is 2016, there are more hard-RTOS prototyping platforms than ever before. My favorite is National Instrument's RIO platform, which lets you use C or LabVIEW (imho the best language for prototyping control algorithms by far). Mathworks also has a platform based on Matlab/Simulink, and the list goes on.
Why use Python when there are existing tools that are made for this type of application?
LabVIEW is great until you need a complicated data structure. I tried writing a tree to do kNN, and it turned out that doing a brute force search was faster even with 100k elements.
This is (now) open source project expecting community pull requests. How many would they get if they went with LabView? The big emphasis that I like seeing in modern scientists is reproducibility of results. Jupyter Notebook is a huge step in that direction (not really applicable here), but just using open source platform is still great.
I agree though that once algorithms are developed sufficiently they should be ported to RTOS platform and this box shouldn't be permitted on open roads.
It seems incredible that this is the source code for the $999 product originally destined to be available this year.
I'm extremely familiar with OBD|CAN and car networks - the embedded code alone is an essay in how trivial something can appear (sending commands to a ECU) without considering the million edge cases that make this a safe product to use.
Because by "works" you mean "I tried it out a bunch of times and nothing bad happened, so must be production ready", because that's what you do when you build websites and desktop CRUD apps and nobody ever died cause the web server choked under load or the web page rendered a bit funny or the request took a full quarter second because the GC kicked off as a wave of requests came in.
And then one day a one-in-ten-million event that never would've surfaced during testing does happen and the OS crashes or the interpreter hangs or some non-determinism causes a period of non-responsiveness or a bit gets mangled because you don't have any redundancies and solar rays are thing or ... and someone dies. And of course that didn't happen in testing, because all of those things are rare possibilities.
And then it happens again.
And again.
And then you're in court being sued for millions. And a bunch of industry experts who do build their systems with redundancies and do design their systems with a safety-first mindset come to the stand and rip you apart for not taking even the most basic precautions. And when it comes out just how many best practices you completely ignored, you're mostly just hoping beyond hope that all of the legal problems stay on the civil side of the civil/criminal divide.
Or, to say it in a sentence, "because the sort of exceptional circumstances that safety-critical software needs to handle with grace are very difficult or impossible to account for with a desktop machine running interpreted code on top of Linux."
I like that you reduced that to a one-liner, and I understand your point. I sure as hell wouldn't hook my car up to this software.
But I also feel like you're being a bit hasty. Obviously a Python script isn't going to turn into a Tesla overnight. But maybe it'll help you find a few bugs before you throw all that time and effort into building the real deal. When I look at a Github repo that claims to be a self-driving car I don't say to myself "yep, looks production ready," I say "Cool prototype, now let's break it." To me it seems entirely reasonable to get a working Python prototype 90% of the way there and then send it off to the OS programmers to design something that actually meets the concept of "production ready".
Yes, I absolutely agree with you on that. Nothing wrong with prototyping in whatever setting is most convenient.
I think w/ self-driving cars there is an interesting ethical question. The full auto cars on the roads today definitely aren't production ready, but they also have constant safety drivers. Probably even ACC systems are tested in the wild with a safety driver before production.
Basically, "is the safety driver sufficient to justify running prototype software in the real world?"
Yeah, that's a tough question that we as a society will have to wrestle with. Knowing that humans are imperfect drivers as well doesn't make it any easier. Even cars today have crippling safety flaws -- remember that one Lexus model that had a sticky gas pedal? Or worse yet, the Ford Pinto. I'm genuinely interested to see how the governments of the world weigh in on this, if at all.
Intuitively, though, I think that buying/using the software is tantamount to accepting its imperfections, so long as they are adequately (factually) presented to you beforehand. You're signing off your ability to make your own decisions, but are still responsible for them.
> You have a lot of solid points, but note that Linux is currently being used by SpaceX in an even more safety-critical aerospace setting
I disagree with the 'more' - how many lives are at risk with a SpaceX failure vs. a self-driving car failure? This is even without multiplying by number of users.
I know how DNNs work -- I've even designed my own.
The existence of a component that is difficult to analyze for safety doesn't justify ignoring well-established safety engineering techniques throughout the rest of the system. That attitude would have us throw out seat belts and snow tires just because the ACC system might be buggy sometimes.
The thing that distinguishes an engineer from a bodger is not just observing what works but understanding why, how, and to what margin of safety it works.
The reason the "realtime" designation exists at all is that, in a realtime system, it is possible to say what the worst-case timing is for any operation, and to guarantee by design what failure conditions can and cannot happen.
You cannot guarantee in a python program the timing of your GC. You can't even guarantee that the whole thing won't fall over with an exception.
And we've already had this problem with the Toyota "unintended acceleration" bug, in the (far simpler) electronic throttle system: https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_... - and that was using C, but not in an appropriate style (MISRA or similar).
Believe it or not - many folks coding out there don't even understand the meaning "real time" in this designation. I recall asking some colleagues in a conversation, "So when you say 'real time' do you mean hard or soft?" To which they replied, "Just, real time, like in real time." I gave up pressing the point when I realized they simply meant "interactive" and didn't know such a designation existed.
Hard real time is really only applicable when designing critical systems. Unless your anecdotal conversation was about one of those systems, it's pedantic to not simply infer soft/firm real time.
I agree. And yes though that is inferrable from my anecdote, in fact I wasn't being pedantic. I actually probably used the terminology wrong myself, but all I really meant was, "what are your latency requirements in this system? How real time?" It was a conversational confusion because they kept using the term and I thought they were using it technically. When I realized they weren't, I didn't bother trying to correct them...
[edit] I'll add this conversation stuck out in my mind simply because I was a bit surprised that no one knew what I was talking about, that's all.
This comment shield should not have been downvoted. Downvoting should be reserved for off topic comments. Someone who is confused and brave enough to ask a question should not be discouraged through downvoting.
The question is naive and has a dangerously ignorant view of what "engineering" means. But it has been properly and usefully answered by others.
(sorry for a meta comment rather than a substantive response, but I feel strongly about this).
you could easily starve a process or thread whose purpose is to stop your car in an emergency over a long enough period of time to cause the occupants harm. I haven't dug into the code so I don't know exactly how this software uses processes and threads (you could have one process no threads and crank down nice on the process) but considering this and GC in python I would not feel confident that this system will always respond quickly enough to emergency scenarios.
The Python control loop is probably the safest part of this system.
The reason that comma.ai was able to get a working prototype quickly (unlike much bigger and well-funded companies) is that they use an end-to-end deep learning model. The model directly learns to issue commands to the car based on raw image data. The problem with this approach is that neural network models are a blackbox. They can behave in completely unpredictable ways.
Human drivers don't run an RTOS and a deterministic bus system either. In fact, I would argue that very rarely do human drivers fail because of jitter or nondeterminism.
This is a weird argument. Human drivers don't deadlock or crash either. Are you sure you understand the criticism being made here? Hard realtime systems make guarantees about the frequency with which code will run on a shared system.
I agree that this is not production ready tech, but still parent does make a good point. I think those two similar comments are downvoted just because people here don't agree with them.
Suppose you make an autopilot in JS running on Electron on WinXP, running on VM on top of Puppy Linux Live CD. And you still manage to prove that your system is 100x more reliable than human driver. Should it be dismissed just because we don't agree with technology stack? The latency of this monstrosity would still be lower than human driver and maybe they would stay lower throughout the operation.
This doesn't have anything to do with latency really. It's about predictability. Given the system you described, there are so many edge cases that can potentially compound, that it is impossible to make any predictability guarantees about the system.
As an example, I worked on an embedded system that controlled electrical motors that was hard real time. The fast task time interval was once per millisecond. No matter what, that got called by the RTOS exactly 1000 times per second. When it didn't finish it's job in time, the result could easily wreck real world items or cause harm to people. Nobody even considered using interpreted languages in that project. The fast tasks all had provably run in much less than a millisecond. That means no loops that could be unbounded, no memory allocations, no recursion, no writing to flash, anything that was even slightly unpredictable was out.
So, even if you could prove that your system caused less accidents than a human driver when it was running well, it would be impossible to do an analysis that defined under what circumstances the system would be running well. Given that, it would not be allowed in a well-engineered real time or safety critical system.
"And you still manage to prove that your system is 100x more reliable than human driver."
That's the rub: how do you prove that? If your software stack is 30 million lines of code that was written by god knows who, I would argue it's nigh impossible without releasing it and seeing what happens, which seems morally irresponsible and legally negligent. If you follow strict rules in coding conventions and algorithms, it's easier to statically verify code is probably correct.
If you read enough security literature, eventually you'll come across the notion of a "trusted system". And your first impression will be that this is a system which is fully debugged and tested and you can trust it to do its job within specifications.
But you couldn't be more wrong. A trusted system is one which you have to trust. No representations are made about whether you should or not.
This is not relevant. If a corporation builds a product that kills people, the corporation is responsible. No corporation is responsible for the behaviour of humans.
No hundred humans are running the exact same OS that can deadlock or crash and as a result drive off the road, which is why this trope about humans being worse than computers at driving is entirely irrelevant to a thread about why it's unwise to try to build self-driving cars on top of Python scripts.
My take of the OP's point was that the deep general stack was unnecessary and decreased reliability. My read was that they think your self driving car software shouldn't also be able to run arbitrary python scripts. My argument is that it's practically irrelevant to reliability. I would not be surprised if in the future you could download an app to your iPhone, connect your phone to your car, and have the app drive your car. Sure, maybe having a specialized real time o.s. without any higher level interpreters could be more reliable in the early days, but the gains from rapid iteration you'd get from allowing cars to be driven by apps running on higher level abstractions would be worth the trade off, and over time the latter could be iterated upon to achieve practically the same reliability as over-engineered systems.
Is there a closed-simulation-environment to test this? By that I mean that the sensor inputs (images of the road, etcetera) are generated by the simulation environment with which the driving agent interacts.
Speaking of which, I've been wondering, if you have digital map data and and a process to generate a 3d model of map data and a renderer that produces photo realistic imagery in real-time, if that could be useful for training a neural network at driving a car by monitoring the position of the car on the virtual road using another piece of software and have it provide feedback to the neural network about how it is performing.
So, ladies and gentlemen of Hacker News, following all the ire shoveled GEOHOT's way for closing up his project, can you please show me any equivalent Open Source contribution from major manufacturers that is comparable? Not like White Paper bullshit. I mean where has Tesla, Audi, or GM shared this kind of platform without Sam Altman writing a think-piece about how they're not as good as Cruise and he knows the truth and doesn't need to listen to fuckin' lawyers.
Sorry, maybe that was an inappropriate aside but I know I'm loud & proud team GEOHOT and have yet to see the Giants in the industry really show they can fend off a determined Goliath.
In Germany you would need a special permission from TÜV (technical supervisory association) for any modification that has or might have an influence on driving behavior. That means that you have to pay them for checking your car and getting a report which allows you to drive it. People are doing this for all kinds of modifications, from custom wheels (where the permission is cheap) to custom engines or suspensions up to completely custom cars (which might get very expensive).
Without getting the new permission you are not allowed to drive it on public road. If you still do then you might also have the problem that your insurance is void.
I don't know what e.g. TÜV would say about a device like this, but I guess the outcome is it would be either to expensive to analyze it or not possible it all since the overall system design and the interaction between the different devices might be not be known (e.g. there might be no public knowledge on how Acura ECUs might react on CAN signals which were never considered during design).
The car would likely still be street legal. (IANAL) However, allowing this system to take control of your car on a public road would not.
Another way to say this is, there's nothing dangerous about the hardware. You could be using it as a logging platform for the data on the CAN bus in your car. But letting the software send commands and take control from a driver is a different story.
> The car would likely still be street legal. (IANAL) However, allowing this system to take control of your car on a public road would not.
IAANAL, but I don't think even that is true. I think it actually is legal to allow it to take control of your car, provided you are still sitting in the driver's seat and can take control back at any time.
> I think it actually is legal to allow it to take control of your car, provided you are still sitting in the driver's seat and can take control back at any time.
Can you GUARANTEE that you would be able to do this in a modified car with such a setup? You might be able to do so if we are talking about pure mechanical overrides. With blackbox software that's imho next to impossible. There will be no public documentation on what the cars systems will do if the receive some commands on the bus from a non-approved device and others from the driver. Even if you can observe through testing that driver inputs win the system might still behave different in some edge cases.
but if you're going down the highway and the AI decides to suddenly swing the wheel far right as quickly as possible, no human being would have reaction time good enough to recover without crashing. I'd be pretty concerned if you could add an autopilot to your car and drive it around on public roads without any sort of demonstration of safety.
> but if you're going down the highway and the AI decides to suddenly swing the wheel far right as quickly as possible, no human being would have reaction time good enough to recover without crashing
"The EPS controller in the car limits the torque to a very small amount, so regardless of the message, the controller cannot jerk the wheel" [1]
But there are restrictions of what parts you can use. I can't just make my own engine and drive around with it without getting that engine certified, right?
This is automobiles, not general aviation. You don't have to get your engine certified to drive it on public roads, but you might have to pass emissions.
Hmm, the usage of .dbc files is interesting. That format is owned by Vector and proprietary. I'm curious as to what the limits of using DBC are, as I'd love to integrate them into the product I currently work on without paying thousands of dollars for the official API from Vector.
George Hotz once said[1], if Tesla Autopilot is the iOS, Comma.ai's system is the Android. Today he's the MAN of his words by open sourcing Comma.ai openpilot.
I read an interesting ethical dilemma related to self-driving cars recently.
Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver.
What choice should the car be programmed to make?
Who should decide what that choice is?
Who is ethically responsible for that choice?
In a normal car that is driven by a human, the choice is obviously made by that human and the responsibility is theirs. But with self-driving cars it's not so clear.
Other threads related to autopilot (mainly Tesla), with great posts that explained in detail why this problem is unlikely to arise in reality and that the resultant action undertaken by the system would be the same in both scenarios (e.g. apply brakes as hard as possible).
IIRC part of the explanation was that a human is unlikely to make a qualified choice in a situation like this anyway, so programming a decision matrix based on the utility of the target into the crash avoidance mechanism is moot. Something like 99.999% of the safety benefit would be achieved by a faster brake response time.
It's also pointless to worry about because the proposed dilemma greatly overestimates the ability of computer vision systems to determine what an object actually is.
These systems are looking for obstacles and road contours. They don't know that the obstruction in the road is a human baby, let alone the composition of an adjacent wall.
Not to nitpick your dilemma, there will always be a dilemma, but at any speed where children in the road are likely (<70km/h, 45mph), the driver of a modern car will have a 90% chance of survival against the wall. Cars really are much safer than they used to be even at the instant where you are meeting the wall.
(Ok, I used a "head on collision" number because I couldn't find a wall number, but the physics are probably pretty close. Numbers from a New Zealand Ministry of Transport PDF from 2012: http://www.transport.govt.nz/assets/Import/Documents/_versio... )
What if those ton of babies will grow up to form a neo-nazi gang that ends up being worse for the world than hitler was? Now we'll have to put future-detectors in self driving cars too.
If it can actually decide, it should probably just decide at random.
Then we can adjust speed limits and move dumb walls and so on to reduce the deaths attributed to random selection by an autodrive system. Kind of like we do with deaths attributed to poor driving.
(I say if in the first paragraph because you need at least 4 conditions to be true: the information to make the choice is available, the vehicle is sophisticated enough to make the choice, there is enough time to make a maneuver and there is not enough time to make a maneuver that saves everybody)
Apply the brakes and stay in the right-of-way. The fact that there is a child in the road is someone else's fault i.e. whoever is supposed to be supervising the child. Chosing to depart from the road would certainly be the wrong choice, since now you might hurt/damage something or someone that is minding its own business and is where it is supposed to be.
To further complicate it...would this be standardized? Would a cheap economy car opt to kill the driver, but a Mercedes try to preserve the driver at all cost?
How much say would insurance companies that insure the automakers have? What about the government?
Also, what level of information about how this logic works is necessary to share with a potential buyer to judge them sufficiently informed?
I know I'd sure as hell want to know if there were scenarios in which my car would decide my life was not worth saving so I could do my best to never even come close to making one of those scenarios potentially possible.
And what if the child's parents have paid Mercedes to avoid hitting their kid? You think you are safe in your golden Mercedes, but not when the kid's got Platinum Protection on his phone!
If it can differentiate between a child and an animal then it's smart enough to add move value judgments such as worth to society (maybe by age). I think the easiest rule is the owner of the vehicle is the most important asset to the vehicle.
