OK, I do that all the time. I overheat easily at temps where most people are comfortable. If it is 20 degrees out and I will be inside a building, especially if ceiling work is involved (heat rises!), I'll wear shorts and a coat and gloves for the little bit when I'm outside. A hat usually keeps you warmer than long pants.
