In certain cases, you know what the neural network should do, for certain inputs, and you have a quite clear idea how each component of the network would solve this, and this should be doable, and with a bit of work you could also construct the parameters by hand, such that it works at least for non-noisy constructed toy input data.
Actually, I think for more complex tasks, having such intuition would anyway be a good idea.
Now, you could use a SAT solver such that it does the work mostly for you. You formulate some constructed inputs/outputs, maybe some other constraints, and let it solve for the parameters. This would be a good parameter starting point for real world data. And if the SAT solver fails to find any solution, maybe your neural network is actually not powerful enough.
In certain cases, you know what the neural network should do, for certain inputs, and you have a quite clear idea how each component of the network would solve this, and this should be doable, and with a bit of work you could also construct the parameters by hand, such that it works at least for non-noisy constructed toy input data.
Actually, I think for more complex tasks, having such intuition would anyway be a good idea.
Now, you could use a SAT solver such that it does the work mostly for you. You formulate some constructed inputs/outputs, maybe some other constraints, and let it solve for the parameters. This would be a good parameter starting point for real world data. And if the SAT solver fails to find any solution, maybe your neural network is actually not powerful enough.