To get the circular polarization, your antenna must generate two linearly polarized waves with 90-degree phase shift.
You can insert a dielectric "quarter'wave" plate diagonally in the rectangular waveguide..
The result will be an elliptic polarization, but by fine adjustments of the plate thickness and length you can get the circular wave. Use Teflon for low loss and on the opposite side calibrate the power level, to be sure you generate a ice circular wave. Good luck!
The rectangular waveguide does not create the elliptical polarization. It guides TE 10 dominant mode and this mode is transformed into TEM wave in a horn antenna.
If you insert the quarter-wave plate diagonally into it, the TE10 mode splits into two te10 modes that if properly phased can cause that an elliptically polarized wave would be radiated from the horn. By fine tuning you can make the wave circularly polarized.
A much simpler structure you can find in satellite TV LNBs. They use a circular horn and waveguide, and the polarization selection is made in mode exciters or transformers at the end of the round waveguide. For linear polarization the coaxial probes are laid in one plane, for circular, they are laid at different distances from the back short.