To offer an alternate point of view, I rarely take any time off, even when sick. It drives my wife nuts because she subscribes to the whole "you will prolong your sickness" camp, but I just don't find that to be true. People get colds, not a lot you can do about it. Allergies are another "sickness" that there is hardly a way to train around. So assuming that I'm not vomiting or losing body parts I think it is all up to personal preference as to whether you hit the gym or not. Aside from feeling under the weather I rarely experience any real loss of performance during a cold. For instance, I had a cold the past two weeks and I still managed to get out and put in 2-4 miles of running a day at a good pace, plus other work. If anything I actually felt better after having done so.

I will also say that I work out in my home gym and don't have those social concerns about whether I may infect someone else.