Sara, lot of things might be the reason. In my personal experience, if you want to make your code work, you have to make sure that each step is 100% correct. Here is some tips from my previous debug lessons:
1. Make sure your FDTD solver is absolutely correct, you have to spend time on this. An empty cavity case is sufficient to verify it. Did you performed any test on your FDTD code?
2. After that do not doubt about field solver anymore, its done job. Focus on your ABC. The good way to test this is use a parallel plate waveguide case, PEC+PMC+ABC along X,Y,Z directions respectively. You should not observe any reflection on this case.
3. Now test if a PEC object in your FDTD can be properly handled, simply put in a PEC box or sphere and check if the EM field inside is always zero.
4. Source, according to your reply, you used a dipole source, NOTE this is completely different than lumped port, port connect two metals, dipole source does not, it simply put fields in. It looks to me that you are not exciting the problem properly, which is very likely why you did not get S11 correct. The reason you did not understand my question "Waveport or lumped port" is b/c you did not 100% understand what you are doing in your excitation.
Let me know your progress.