I've ridden in the 20's before going to work in the morning. Got two baraclavas on...thermal, tshirt, thermal, hoodie, leather jacket. That makes my body decently warm, well barely, but only have the Knox Handroid for gloves. Hands are frozen and numb just after 5 min of riding. And it takes me 25 min to get to work.
Will have to try the latex glove thing.