as far as i know, ADS tool provides all functions to calculate BER and power levels of received signal.
to determine BER in signal, u can add CRC checksum or any error detect coding at the end of data packet and at receiving side after demodulation, u can use the CRC checksum or the code to determine the error in received data stream with how many bits error. Once u found no of bits has been changed, BER can be given as (no of bits changed in data stream/total no of bits in data stream).
And FM is an analogue modulation technique while FSK is digital modulation technique. If u have both modulator and demodulator implemented for FSK designed either in application software or hardware then u r done, u can simply call API's or send modulated signal to that particular hardware. If u have urself implemented in software the FSK demodulator, send data to ur implemented function as parameter. FM demodulator if fed with FSK signal will return different analog voltage levels as output instead of bits or symbols depending on What FSK modulation technique u are using, for e.g. BFSK, QFSK, so it is better if u use FSK demodulator.
Hope that helps.