Distillation will still work if the inputs are slightly entangled with each other or with other qubits.

I recommend just simulating the specific case you're worried about. It's only a 15 qubit circuit; not at all expensive to check. You'll either see it working and stop worrying, or have an amazing concrete counter example to publish.