I've been running Debian and U untu chroots on all of my Android devices going back to the G1. It actually works very well. One key for me was making sure to bind mount the sdcard inside of the chroot environment so I have access to all of my local files. As far as X goes, I find the vnc solution works okay provided you aren't trying to watch videos but for programs like Writer and Geany, it's fine. The X server you can download from the market needs a lot of work to be useful but hopefully that'll happen.