Try DisallowTCP = false under [Security] in /etc/X11/Sessions/Gnome/custom.conf or similar for other window managers. Obviously this isn’t a very desirable solution, emerge --emptytree gnome might do the trick too.
(and obviously this might be just one of the many underlying causes for the very generic symptom of X not accepting connections)