Short answer: You need to increase R3 to about 27K.
Long answer: When Vpd is negative, the input pins of U1 are at ground potential so Vpd is reduced due to the current flowing through R1 and R2.
If Vpd were provided by a perfect voltage source, there would be no problem. However in your circuit the source impedance is R5||R4. That forms a voltage divider with R1||R2 for negative inputs.
For positive inputs, the voltage at U1's input pins is the same as Vpd, so there is no voltage divider action due to R1 and R2.