So you are effectively letting LO, LO+IF, LO-IF, + a bunch of NxM mixer products into your X2 doubler? Not so great a system design! The X2 multiplier, by its nature, will have a poor IP3 and all those products will cause 3rd order products at its output.
Also, there is a huge 2IF x 1LO product out of the mixer since the IF covers more than one octave.
If you really need low spurious, you might need a tracking filter, or a switched filter bank, between the mixer and the X2. Or put the X2 in the LO line only, maybe another x2 with switched filters on the IF line before the signals get to the mixer.