Recently I was also facing the same problem. But I got on my own on how to use 'phaseMargin' function in Cadence ADE Calculator version 5.1, which displays direct result (no need to subtract result again from 180) on ADE Outputs window. May this reply help someone in future.
** These steps are for open loop AC analysis, not stb. Please follow the below steps and let me know if you can't get it.
Note: Assuming you've already run the simulation once.
1. ADE -> Outputs -> Setup
2. Using expression entry: If you've labelled output in schematic as 'Vout', enter the expression as 'phaseMargin(VF("/Vout"))' without single quotes.
3. Using ADE Calculator:
3a. Instead of entering expression, Open Calculator from 'Open' button.
3b. In the calculator window, click on 'vf'
3c. Select output node in schematic.
3d. Return to Calculator window.
3e. Select 'phaseMargin' function under 'Special Functions'. Expression will be displayed.
3f. Go to 'Setting Outputs' window in 'ADE'.
3g. Hit 'Get Expression' button.
4. Add it to the 'Table of Outputs' with name (optional). Hit 'OK'.
with regards,
Praveen