Testing validity of comparing two C# types and caching the results

I'm trying to write a generic WPF converter that will allow me to do logical comparisons (operators <, <=, >, >=) between any two objects. The end goal is to be able to compare in XAML two integers, two DateTimes or two of anything for which it is valid to do so. This might even involve two different types (e.g. integer and double)

The major difficulty in implementing such a converter is to first know if the comparison operators are even valid for two given types. This isn't like equality comparison, where every single object has an Equals function. So I need a generic way to figure it out. Fortunately I found one in the accepted answer to this question and it looks good.

However this approach is exception driven; If a comparison is invalid it throws an exception. It seems it could be quite slow. I'm not sure but it seems that way. If so, I don't want to have to test if it's valid to compare, say, two integers every single time it is asked do so.

My questions are

1. Given that most uses of this converter will naturally involve valid comparisons (which will not throw exceptions), is my concern about speed justified. (Note that in my use cases, the converter could be called many many times rapidly).

2. If so and I were to try to cache the results, what is the way to go about it? I've got 3 elements that make each comparison unique: the type of each of the two operands and the type of the comparison. I thought of using dictionary with a key of concatenated String names of the 3 elements. Is that reasonable or there something else better?