I do this too. Push ups, Dips, and squats a few times a day in my office. I feel it keeps my overall fitness level up, as well as energy level. Its a great way to "wake up" when your feel yourself falling asleep at your desk.

I do take days off from this when I need to recover from a hard day at the gym. But overall it doesn't really affect my recovery.