I suppose it depends on how long you have to get used to it.
Magellan remarked on how the natives of Tierra del Fuego walked around naked in a climate where it rarely got above 50 degrees Farenheit in the summer (cite).
On the other hand, hypothermia usually develops between air temperatures of 30-50 degrees (cite). Assuming one is dry, and there is no wind, I suspect you would begin to develop symptoms closer to 30 degrees than 50. Much depends on how much insulating subcutaneous fat you are carrying, and how active you can stay. Also what the floor is made of - a bare stone floor is going to conduct heat away from you better than carpet. If the water that you drink is colder than body temperature, that will also affect how fast you cool off.
At a rough guess, an un-acclimatized person with normal levels of body fat, in a room large enough to allow walking and with an insulated floor, could survive temperatures in the 40-50 degree range as long as he/she was able to keep moving. Ultimately, s/he would be exhausted and hypothermia would begin to develop.
I suppose it is inevitable to point out that you are not the first person to ask the question, or one like it.