A half-formed theory, based solely on heresay, conjecture, and anecdotal
evidence:

People with higher novelty-seeking get more pleasure out of physical
exercise. People with lower novelty-seeking get less pleasure out of
physical exercise. I haven't yet decided why, although it could be a result
of the effect of endorphins on an addictive personality. I have low
novelty-seeking, which is why I hate exercising.