DisallowTCP = false under
/etc/X11/Sessions/Gnome/custom.conf or similar for other window managers. Obviously this isn’t a very desirable solution,
emerge --emptytree gnome might do the trick too.
(and obviously this might be just one of the many underlying causes for the very generic symptom of X not accepting connections)