yeah man, take a few days and let your body recover. your immune system is working over time right now, and thats gonna consume alot of energy, then add to that micro trauma to your muscle cells and your body just isnt going to recover easily.
as for the wieght, I would venture you arent losing muscle just yet (atleast not a lot), what you lost will mostly be water, and maybe some glycogen, nothing to be concerned about. Just take it easy and all will be well by next week or so.
I don't do this for my family, my friends, women, accolades, pride, or ego. I do it for me and no one else, its just part of who I am.