Im not to familiar with ford, but I know a 2.106 valve is rather large and I think that is where most of the ptv issues lies.
Intake and Exh are ~.10 bigger than LSx stuff. Not super huge, I wonder if it has to do with the VCT ? I'm not sure how far it will advance the cam but that might be some of it too. I wonder if you got the XT cam and could lockout the VCT and install it straight up or a few degrees in either direction if it would work. I know this defeats the purpose but just for discussion purposes.