The symptoms are exactly like this bug and it only occurs with xl.
In addition to that, it only seems to occur with a secondary graphics card being passed through.
I end up with a null domain that I can't shutdown.
Killing the qemu-dm process with kill pid sorts that out though.
I'm running dom0 kernel 2.6.32.50.
Matt