I think the diode-at-output-of-supply is the simplist and best option. Schottky power diodes should be used. If the diode can be inserted inside the voltage control loop, then you will see so diode voltage loss at the output.
With this method, only the higher voltage supply will provide all the power to all your boards. In your case the 5v supply. Only when the 5v supply fails/ falls below 4v, will the 4.5v supply takeover.
Without the diodes you would need to switch on/ off a supply based on monitoring the rail voltage. Thats a tricky business at best.
In case you were thinking of using these supplies in parallel to simultaneously provide power, then a more complex scheme is required which balances the voltage from both supplies.
In all cases, any actual transition or switchover could cause a glitch in the power rail which might affect your boards functioning. So plenty of caps are advised.