option 1:
as you said, to modulate the bias current of your source follower, but I suggest the best way is to design a source follower at 300ohm, and duplicate 6 of them in parelle to match well. then you can choose to turn on 4 of them to achieve 75ohm or 6 of them to achieve 50ohm.
option 2 :
1st design a source follower like you did, target the output impedance at 50ohm.
then, put a poly resitor under the cascode device source, in serial, at 25ohm, and you can use a mos switch device (make it big) to short the both end. so turning on the mos switch, you got 50ohm, turning it off, you got 75ohm
note that bias current change with temperature and vcc, so does Gm. It is very hard to maintain a constant output impedance across corners. this has been a major headache for us. if you really need accurate impedance, you need a lot of compensation techniq to do the job.