I think it's still hard for me to turn down work if it's really good because for so many years I was so desperate to get a job and couldn't and so it's kind of an anathema for me to turn down work.
The key is to keep company only with people who uplift you, whose presence calls forth your best.