It's not really failing silently, it's telling you through the overflow flag. To me this seems logically consistent with how one would expect overflow to behave, as it does with other instructions like ADD.
That said, I think this instruction would be safer and more useful if it still set at least the remainder result bits (which should always be valid). Then this case would not require checking, as would some other common cases like 'execute every odd iteration' kind of code.
Does ADD on m86k leave the output unchanged or truncate? If division truncated the multiplication result then the moduls would still be usable and this bug would not have happened. Not performing the operation at all on overflow but not raising a fault is odd behavior to say the least.
That said, I think this instruction would be safer and more useful if it still set at least the remainder result bits (which should always be valid). Then this case would not require checking, as would some other common cases like 'execute every odd iteration' kind of code.