For purely digital simulation, from Encounter, you would run rc extraction from within Encounter - and this will allow you to output a SDF file and Verilog netlist you can simulate in ModelSim/Incisive. (You can also generate a SPEF if you want to look at the RCs, but you need a SDF for simulation).
For analog simulation, you need to extract a spice netlist from your GDSII, using Cadence QRC / Synopsys Star RC / Mentor Calibre xRC, then sim in hspice/spectre