Altium: applying rules to polygons on mask layers

I've got a component which requires a fairly hefty heatsink pour on the bottom layer. Next to it I've got a couple of high voltage traces that I want to keep well clear of the heatsink polygon. I want to make the poly as big as possible, so instead of manually stepping around the traces in question I just gave them a 1mm clearance rule. This works fine - creates a nice neat cutout around them. The problem is I don't want solder mask on this heatsink, so I copied the poly and pasted onto the bottom solder layer. Since the bottom solder is not an electrical layer, the rules don't apply to it, and the poly goes right over the vias in the HV tracks (see attached image). With no mask there, the solder could bridge over to the via and cause much bigger problems. I know it's unlikely with a 1mm gap, but I've got other locations with the same problem with 0.2mm gap. I've tried naming the polygon, and giving it a net and making clearance rules for those but nothing seems to work. I've also tried copying it from the bottom layer and pasting to the solder mask without repouring, but the primitives stay on the bottom copper layer until you repour it. Short of manually laying it out, is there any way to apply clearance rules to polygons on non-signal layers?

Solder mask is a "negative" layer, so any objects placed there will cause an opening in the soldermask. I don't know of any way to do what you want with rules. With parts that need heatsinking, I put a base-level heatsink in the footprint, and I manually add to it as space allows.

FYI I posted a note about making footprints with tented thermal vias in Altium: https://www.edaboard.com/threads/205944/ You might not be trying to do that, but there's a similar concept involved - in order to make an island of soldermask where you want it, you need to paint around it, you can't paint the whole area and then somehow force soldermask inside the open area. FYI I have been able to do things which made it look OK in Altium, but it was still wrong in the Gerber files.

Thanks for that - I was wondering how the hell to meet the chip mfr requirement that the vias be tented on the top to prevent thieving, and with your guide I've got that sorted. I've worked around my problem by making the bottom layer poly put octagons around pads instead of arcs, which let me manually place the solder mask poly over the top of it fairly easily. It's kludgey, but it'll do unless someone comes up with a better solution

On a related note, there are two conflicting goals with the design of the heatsink - one needs to maximise the heat transfer during operation without making it so good that the pad can't solder during reflow. Our fabricator doesn't have an x-ray machine, so we can't tweak the reflow profile much on the fly. One of the app notes you posted says that the thermal pad shouldn't have thermal relief, but do you have any thoughts on how to approach the mid layers and bottom layer heatsinks? At the moment I've got a solid pad on top with tented vias and identical polygon pours on the other three layers with thermal relief connects to the vias (0.3mm clearance, 4x0.3mm connects). I've got three different high-power QFN chips in my current design, so I really want to get it right.

I recently did some designs with thermal vias under the thermal pads; the thermal vias went to copper on the bottom side and in some cases also to the ground plane layer, with no thermal relief. I was worried that the assembly house might have trouble with the soldering, but the boards came back looking great. I asked them if they had any trouble, and they said no trouble at all. I'm not sure what they did, I guess they used plenty of preheat, but the soldering was fine. It's not in volume production (yet), so just a few dozen devices soldered so far.

On another board, with no ground plane, there are thermal vias to copper on the bottom side, and we've had over a thousand devices soldered. Vias were not tented, and even though the solder paste was patterened to keep clear of the vias, on a few devices there was some solder wicked through the hole to the other side. The solder joints were fine, but the presence of solder on the bottom side caused a problem, for the following reason: These boards have no through-hole components, and are attached using pressure to a heat sink with a thin adhesive thermal pad. A solder bump on the bottom side can, and in one case did, perforate the thermal pad and cause a short to the heat sink. That's when I had to go back and figure out how to tent the vias in the footprint, and posted that note.

How many volts are the 'High Voltage' traces carrying? as 1mm only covers up to approx 150V, I would look at any regulations relating to SELV where the product is going to be used.
Thermal vias are best with no thermal relief, infact adding thermal relief makes them worthless thermaly as the thermal resistance increases greatly and thier ability to transfer heat away from the device becomes very limited.. As a board is reflowed, ALL the board and components are brought up to the same temp during pre-heat, this avoids thermal shock, thermal gradients and allows soldering of not just normal SMD joints but also solder mask defined pad joints where there is a lot of copper attached to a pad. Thermal relief is geared more at wave, hand or selective solder, where the thermal pattern for the process is different from reflow, and for through hole components.