Because you have more control over your hobbies. At the end of the day, the success of your career is (at least in part) dependant on luck and the decisions of others.
I think most people are more likely to work at a job they dislike and spend their weekends on their own endeavors than the other way around, that's all.