p1db op1db
The ideal balun (what is the impedance ratio?) and 50 Ohm source/load works if the Circuit under test is matched both at input and output.
If not you may see big differences plotting voltages (what you have done in the case of Gain) and powers (in case of IP3 and P1dB). From your simulations it seems that it is the case (voltage gain measured from first plot is really different from voltage gain measured in third plot). This is maybe the problem...
Try to plot the single end to single end gain, maybe you'll see something different, or with the same bench try a PSS+PSP (s-parameter analysis, that takes into account power and not voltage), plotting S11,S21, S22 (input match, power gain, output match).
I hope it can help.
Mazz