Personally, I think it's a UI issue. The concern would be mitigated if the identity selector were to simply preface the display token with a caveat:
The following attributes are what the IDP claims to be sending. If you do not trust your IdP, do not click on "Send".
If the UI doesn't misrepresent the reality of what the DisplayToken is (and isn't), then we're hunky-dory.
And of course, CardSpace is not the only WS-Trust based identity selector in town. The other selectors are presumably under no constraints to deal with DisplayToken in the same way as does CardSpace?