Then workout after a small breakfast. I am in the same boat and have read sooo many articles and text for both but both side agree that as long as the end of the day, you burn X more calorie you are golden. So if you eat like 400 calorie and then workout, just make sure to burn 800 (well that is what I tell myself--the breakfast is the reward that I have to earn).
In short eat the small breakfast but keep the overall calorie in defecit.