Certificate services just plain refuse to work properly in Server 2008 R2, with a configuration that worked just fine with Server 2003.
Finally figured this out, this has been a crazy problem. First I'll tell you what it wasn't:
Wasn't local firewall stuff (but if you're across and FW/VPNs you should still check for RPC filtering like with ISA/TMG)
Wasn't the local security group, and as a note the group is no longer called "CERTSVC_DCOM_ACCESS", in 2008 R2 it is called "Certificate Services DCOM Access"
Wasn't any permissions issue
Wasn't a DNS issue
Wasn't a DCOM or DTC issue (not exactly)
So I figured I would install this on one of my DCs, no dice, and it broke all kinds of strange things, TMG EMS, WSUS, weird RPC errors for Exchange 2010, etc. Turns out that the Certificate Services Role does NOT work on a 2008 R2 domain controller (I tried it on 5 of them). In every single case, I got the dreaded (and vague) RPC error. I'm not sure what is conflicting, because even the debug logging is terrible with this product. I am surprised that MS didn't catch this one as it is a common scenario and a big problem for people all over right now.
As soon as you remove this role from your DCs and install it on some other non-DC server everything works perfectly!
keywords: RPC errors, 0x800706ba, TMG EMS, certificate, CA, root CA, DCOM, COM, COM+, DTC, KDC, CERTSVC_DCOM_ACCESS, (WIN32: 1722)