To measure the AC move the earth to the other side of the bridge rectifier (you can join one terminal of both windings together as shown attached). the wave will still look bad because of the inherent inductance and resistance of the windings, the greater the load the greater the distortion.
The coupling factor factor just simply gives the transformer ratio i.e. 10:1 so 100V input (200V pk to pk) = 10V output (20V pk to pk).
The DC volts will be half the pk to pk volts minus the diode volt drops, the transformer output voltage will reduce slightly with load.
Im no expert on transformers so if anyone reading this would like to chip... The DC output with just one cap is very crude to say the least and complex filters can be built to "smooth" the DC voltage, however lumpy DC is fine for some applications like motor control or electro/mechanical switching, but if you intend on powering sensitive components then just add a reg to the output, for small loads (up to 2A) a linear reg will be fine.
with a 100VAC input this transformer will give around 6.2VAC output at about 0.7A:
MULTICOMP|MCFM70/09|TRANSFORMER, MINI, 7VA, 2 X 9V | Farnell United Kingdom
Add a bridge rectifier, cap and 5V reg and your ready to go.
Dont forget to fuse the DC and AC side.