Of Ashish's items, the first three (and presumably their placement reflects priority) are usability issues, e.g. too many clicks, too slow, and confusing UI. (I have my own nit.)
It would be fascinating to learn what Microsoft's own usability tests for Cardspace have unearthed. Do users like the card metaphor? How many cards before they get confused? What is the best way to help users choose a card in a given context when multiple cards are applicable? How do users react if a Cardspace authentication fails?
Have such tests been done? Or for the other compatible card selectors?
Such a report would have made a great submission to DIM 2007.