You can't spot reduce fat. The only way to lose fat from an area is to lose fat over all. Weight doesn't take into consideration the muscle to fat ratio. You can be the proper "weight" with no muscle tone and fatty spots, or you can be the same weight with more muscle and less fat.
Now when the doctor tells you you're at the right weight, he's probably just looking at a little chart that doesn't take into consideration any of this - or anything else. Weight is a very flawed thing to go by.
So, if you don't like the fat on your calves, lose some fat through a deficiency in calories and exercise.