Once VREF falls below a diode drop, Q3 shuts off and is high impedance. Basically, the VREF node will be clamped at a diode drop with respect to Q3/ With the VREF node being high impedance you have no way to discharge that node, excpet for leakage current. It will eventually discharge if you give it a long time. If you put a small load on VREF, it will discharge to zero. If you need it to discharge to zero immediately when VIN collapes, you'll have to put some sort of pull down on that node. There is nothing wrong with the circuit as it is.