A very commom cause of headaches is dehydration. Obviously you'll lose some fluid as you sweat, but there are other losses too. If you live/work/work out in an air-conditioned environment you'll lose some by every breath as AC usually lowers the humidity in the air.
Then there is the question of what you're drinking, coffe and tea for instance have diuretic properties, which also can have a negative effect on your fluid balance.
Part of the trouble is that if you wait until you get thirsty you're actually too late (Not that it's particuarly critical, but still), and if you drink too much in one go your body won't be able to absorb all of it.
If you really want to treat your body right you should try to drink small gulps frequently, particularly while exercising.
Then again, if you're fully hydrated when you start working out you should be able to go at least one hour before dehydration actually becomes significant.
Checking one's level of hydration properly isn't that easy. The only way that I know of that is readily available way is the slightly unpleasant method of checking the color of your urine. If it's as dark during the day as it is first pee in the morning, then you're quite probably dehydrated.
If dehydration is what ails you, try drinking 500 ml of water the hour before starting your workout and see if that helps.
Other reasons? Well, some gyms play music real loud - that can also trigger headaches.
Maybe you're tense and nervous about going to the gym or the aerobic class? That can trigger tension headaches.
Maybe you actually need glasses, but leave them off while training? Or you're working out outside, in bright sunlight w/o sunglasses? Squinting is another possible headache trigger.
Finally there are the more "medical" reasons. A workout will affect your blood pressure, pulse and whatnot, so although at the far end of the probability scale there might be something there as well.
But if there aren't any other indicators, and the headaches are neither severe or long-lasting I'd go with the dehydration theory without a doubt.