serial terminal emulator bray
OK, with your test equipment limitations here is what I would do.
First, read the RS232 standard so that you are familar with how it works. A serial port and RS232 are not the same thing. The RF units you bought want TTL levels. The levels coming out the back of your computer are RS232. There are several circuits and chips that will convert between RS232 and TTL.
https://en.wikipedia.org/wiki/RS-232
Next, make a loopback plug for your computer like is detailed at:
Now open a terminal emulator. There are lots of these. All windows computers ship with Hyperterminal which will work for your purposes. Some people prefer TerraTerm, Bray's Terminal, or other terminal software. You can use any. Open the terminal software, configure it for your baud rate and attach the loopback plug. Start typing and note what appears in the terminal window. If two copies of each character appear, then look through the setting to disable local echo. Now characters should appear only when the loopback plug is attached.
When this occurs, you now have a basic test bench for testing your serial communication. However, this is working at RS232 levels. Therefore, now get a chip like the MAX232 or some other 232 transceiver and attach it to the serial port pins 2 and 3. On the downstream side of the chip, again loopback send and receive. Run the terminal program and verify that the characters are properly echoed.
Finally, you have a TTL level test bench and can attach your RF modules. Connect the TX module to the TX line and the RX module to the RX line. Use enough wire that you can get the modules some distance apart. Now start typing in the terminal program. The characters that appear on the screen will be those sent across your RF link.
Most likely some characters will work and some will be garbled. FVM and I suspect that capital "U" will work best as it is a balanced signal. Make a list of the characters that work best and those that often fail. Open an ASCII chart and verify the sequences of the failures. It is suspected that the failed characters will have several consecutive bits of the same logic level.
If this is the case, then the solution is to encode the data before transmission to equalize the logic level transitions. Then on the on the receiving end decode the data before it is used. There are many methods to do that. Manchester encoding is one of them. If you have several characters that always transmit fine, you could also just use these characters as logic bit descriptors. For example, lets say that "U" and "F" work fine. You could assign "U" to represent logic 1 and "T" to represent logic 0. Then to transmit the hex number '0A' you would transmit, "TTTTUTUT". Of course, this crude method is 8 times slower than other encoding methods, but it works fine if the volume of data is small.
Notice, that I did not use an oscilloscope. My using the computer to construct a systematic test bench, you have all the hardware you need to troubleshoot this. Finally, as a side note, the PC parallel port can be used as a low frequency logic analyzer. Search the web for the code and the hookup.