Forget fat loss for now, you don't have much fat but you might look a little 'soft' becuase of the lack of muscle. If anything I'd suggest eating more and lifting. With more muscle on you you'd look fine in no time. You're not in a terrible state, just lacking some muscle but that's easy enough to rectify.
Read the weight training 101 sticky