From my pointof view, we are constantly looking at the frequency response of an amplifier. We look at it most often in simulation, and in that case we are looking at a log-log plot of gain (vout/vin) and phase (phase of vout/phase of vin) on the y axis, versus frequency on the x axis. In this plot if we have a one pole system, the gain is rolling off with a slope of -1. The location of the pole shows the frequency where the gain breaks from a 0 slope to a -1 slope. This means that for each decade increase in frequency, your gain will fall by 20 dB. This fall in gain starts at the pole location (frequency). So your gain plot will be shaped by the pole locations.
As frequency continues to increase beyond the first pole, if we reach a zero, the gain slope will change from a -1 back to a 0. In other words, the gain will stop falling with increases in frequency.
Phase will shift more gradually, with output phase lagging the input by up to 90 degrees for each pole and reversing (leading, or reducing the lag) when a zero is approached. In this point of view, poles and zeros shape the gain and phase response over frequency. I don't see how it is helpful to think of gain being independent on the pole and zero frequencies.
In my opinion, books that show -1, -2, -1, 0 etc slopes of gain on a log-log plot are most helpful in understanding the frequency response of systems built around amplifiers. As much as I have loved thinking about the complex plane, it's usefullness in practical work has been limited.
Hope I have not misunderstood your question.