I'm sure there's a relay in there, but if it does, it won't look like a standard relay.
Baseboard heaters often have a resistor-typr relay in them. Let me explain how that works:
The thermostat is usually an 18vac circuit. When the t-stat closes, current flows through the resistor, making it hot. This heat causes the contacts to close, and the heater begins to make heat.
Because of this method of working, there is a significant delay between the t-stat closing, and the relay itself closing. You can usually hear a 'click' when it does.
I found it most helpful to place a thermocouple, or thermometer, in the heater fins, so I could readily tell if the fins were warming up or cooling down.
Another variation will have a t-stat mounted on the face of the neater, with a 2-pole switch right behind the faceplate. These switches are misleading, and very easy to mis-wire. I reccoment using your meter to ensure you understand which contacts belong to which wires.
