People sometimes hold themselves back because they want to use racism as an excuse for them not being able to achieve what they want to achieve.
It's often difficult to slough off all that we've acquired, all the comforts and safety nets modern life provides for us, and realize that in those days, people were living very much on the edge - life was incredibly hard!

