Apart from it's DC bias function, the ethernet magnetics mainly provides common mode isolation. Both sides are 100 ohms differential transmission lines. Impedance matching wise, the magnetics are almost transparent. There are two simple conclusions:
- the on-board part should be good 100 ohm matched differential pairs. The overall losses aren't affected much by the magnetics placing.
- isolation of external common mode interferences suggest to place the magnetics near to the PCB border respectively the connectors.
There are possibly contradicting points, e.g. suppresion of PHY born EMI. But on-board interferences can reduced by using high quality parts, off-board interferences can't be controlled essentially and it's difficult to prevent them from coupling to the ground plane and adjacent signals.