The caps in series with the AC input must definitely go.
The ones across the diodes can help, but they should be ceramic type and only about 4.7nF.
The Tantalum caps are fine. They should be placed as close to the chip as possible.
But I question the 1000uF cap on the output side. It should not be necessary. The output cap helps with the transient response, but the ripple rejection is done mainly by the IC.
The IC has a minimum rejection of 41dB, that is about 100 times. So for 1mVpp output ripple, you must have less than 100mVpp input ripple. With the 2200uF cap at the input, the ripple will be about 300mVpp at 100mA (max regulator current). So if you want this kind of performance, you can increase the 2200uF three times, or build a PI filter at the input (use the 2200uF right after the rectifiers, follwed by an inductor and another cap, say the 1000uF, which you move from the output). The whole idea is to keep the input ripple under 100mV, if you are happy with 1mVpp at the output.
If even better performance is required, then you can cascade two regulators, say a 78L08 (giving you 3V headroom for the 78L05, but you need at least 11VDC from the rectifier), followed by the 78L05. Thus, the ripple will be reduced by about 80dB overall, that is 10,000 times. So getting less than 1mVpp output ripple(at 120Hz) should be feasible.