No, your understanding is not correct.
The aim of impedance matching is to maximize the transfer of power from the source to the load. This is accomplished when the load impedance equals the complex conjugate of the source impedance. In case of real impedances when the load resistance equals the source resistance. For examples when you have a generator having 50 ohm output resistance (series to the generator) and a load of 50 ohms (from source resistance to load).
But when you need to transfer such a power through a trasmission line you have to take into account that also trasmission lines have their characteristic impedance (typically real: 50 ohm, 75 ohm, 32 ohm,...). To maximize the power transfer this last have to matches with source and load impedances. The impedance of the cable doesn't have to be consided as in parallel with the load impedance; it just a "transfer impedance". Mimatches of one of these three values will cause reflections that leads in many effects you can learn on a book.
The connector impedance has the same meaning of that of the cable: the connector behave like a transmission line.
So, in your example, you will have a generator with 50 ohm output impedance connected to a 50 ohm cable to a 50 ohm load. Everything is 50 ohm. Of course I'm neglecting al other effects present in the real world.