But what about the vulnerability as an indicator of the general quality and reliability of the tool? Casting doubt as to the ability of the tool to generally be accurate and specifically maintain proper chain of custody documentation would seem to be a reasonable legal defense tactic.
I would expect a defense lawyer to say something like
"The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? If a police officer were unable to tell the difference between his or her own thoughts and things he or she had read, we would dismiss him or her as a reliable witness."
That argument has never had the lightest effect on the (ongoing) use of polygraph testing by law enforcement agencies, nor on the use of polygraph results in obtaining convictions by courts. And yet it is well and truly established that polygraph testing is snake oil.
>I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering?
To be clear, all you accomplish with that statement as a defense attorney is that you didn't get a credible enough expert, as any Computer Scientist should point out that is the fundamental character of the Von Neumann computing machine architecture, the very model of computing that most computers are designed according to, and most programs are written to run against. They would then further expound that software development had developed methods to mitigate this problem, which minimize the llkelihood of such architecture quirks being exploited, and most certainly leading to a state of affairs where any such vulnerability could be identified via a source code audit. This would open the door for the defense to require the prosecution to produce source code for their tool to prove to the court whether the vulnerability exists or not.
A good defense would then follow up by asking whether or not there was some way to detect whether there had been a successful exploitation on a device. "That's where things get tricky", the expert should reply, "because if arbitrary code can be run, given enough time, someone could cover their tracks successfully. It is plausible a mistake could be made in terms of the implementer of the exploit missing a timestamp, not properly serializing something, not cleaning out a log that could be then reconciled with something else, but the possibility of a completely clean alteration given enough time and resource was still on the table.
The prosecution would then endeavor through chain of custody logs, affidavits, data on the device, possibly comparisons to other cases convince the jury this is all hogwash, and the defense is grasping at straws, and ultimately full of shit, without tipping the defenses hand that if this case is in question, other cases may be.
Mind the brilliance in Moxie's actions is not that he'd get someone off the hook, but that he's now forced prosecutors into a position where if they want to rely on Cellebrite data as a lynchpin of their case, they have to open the door to public scrutiny of the implementation. Of course, this will just be mitigated by law enforcement ultimately engaging in parallel construction anyway.
Or, Cellebrite updates/audits their software to mitigate the vulnerability, or re-implements it on a non-Von-Neumann computer.
Again, not a lawyer, just read some stuff on how to think like one once.
Again, I suggest that if we want to understand how this stuff plays in reality, we'd do better looking to examples of how arguments like this have fared in previous cases, rather than trying to reconstruct this case from first principles. The idea of arguing against the reliability of computer evidence is not a new one; nor are vulnerabilities in forensic software (or, for that matter, the existence of very important major commercial forensics tools that defendants could know about and do vulnerability research on).
Here, by way of example, is the Grugq talking about this idea twenty years ago (presumably: about EnCase).
I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? If a police officer were unable to tell the difference between his or her own thoughts and things he or she had read, we would dismiss him or her as a reliable witness."