Hi,
When defining the boundary conditions, it is important to surround the air box (or it can also be the vacuum box) by a radiating conditions that enable the wave to radiate infinitely far into space.
But Ansoft documentation does not precise how big should be the air box.
I know that a too small height for the air-box with a surrounding radiation boundary does not enable the wave to radiate correctly before touching the boundary condition.
You can also choose a minimum height according to the size of the waveport. There are rule for that which depands on the dimensions of the transmission line.
For instance, the minimum port height is 4 times the substrate thickness for a CPW line.
What is the type of transmission line you use ?
5 mm is not too big for me.
You can also plot the mesh. You'll see that the mesh is less dense in the air box.
You can also go step by step by increasing gradually the box dimension and looking on how the S-parameters vary, especially the insertion loss.
I Hope it will help you,
Regards,