The low supply rejection of the error amplifier contributes most to the supply induced disturbance of the LDO (If the bias current you get is clean., that is).
If you supply the error amplifier from the output of the LDO, which is free from the supply noise to a good extent, the output of the error amplifier's dependence on the supply voltage is further weakened.
The attached figure presents a simplistic view of this.
It needs startup because, at zero state (no currents), there is nothing pushing the amp to work. There has to be some voltage at the output of the LDO to make the error amp work linearly.
The diode connected transistor can be replaced by a resistor as well.
If the last stage of the amp is designed as a folded cascode stage, you would not require a startup circuit.
This would make the LDO PSRR high, but then the output's supply dependence will be limited by the PSRR of the input bias current. You would then need a similar technique at the bandgap block to get to the desired spec.