Well, aside from the fact that you are subject to the laws of diminishing gains from the minute you pick up your first weight... if you weigh X, and eat calories to support Y (eating over maintenance), after you weigh Y (or even as you are approaching it) doesn't it make sense that you would need to start eating cals to support weight Z?

So Y-X gets you the amount of cals available for growth, but as X increases that number gets smaller and smaller. So while it was significant enough to build bricks with before, now it might be making legos. Keep it up long enough and it might be making grains of salt.

Takes a long time to build a mountain from grains of salt...