It could have to do with that xterm is trying to faithfully emulate an actual terminal (viz., VT220 with extensions) by replicating the VT's internal state machine, whereas libvte is trying to approximate the behavior of xterm and cutting corners as it does so.

Note that while libvte seems to have greater throughput, its latency is terrible compared to xterm.