I've heard and have actually observed the same things about swimming. When you do a head to head comparison, swimming just doesn't cut the mustard in terms of fat loss. And you have to make sure not to consider elite swimmers in this. If you tend to have a lot of extra body fat....you will never be an elite swimmer, most likely.
I've actually seen a couple of articles touting swimming as the IDEAL way to lose fat. And then I discover their reasoning is because Olympic swimmers have low bodyfat levels. LOL. Of course they do. You can't have a big belly dragging through the water and be an olympic swimmer. Skill level may be a factor also. I know swimming would suck for me cuz it would take me too long just to become proficient let alone burn a ton of calories. Then you have the cool water bringing on fat as an insulater but I don't think as yet that theory holds water
.
There are all types of theories as to why but to me the most compelling is just the studies comparing swimming to other things where swimming always comes out at the bottom and just observing overweight or slightly overweight people who swim a lot of laps.
I agree with Sleazy. Do some swimming if you like it. But I wouldn't choose it as the preffered cardio activity. In any case variety is good.