My mother also had us take piano lessons, and this had a similar effect. I hated those lessons, but I now play regularly for pleasure and have even tried my hand at composing.
Are there challenges? Absolutely. But in aggregate, this is a very strong and valuable company.
In a memoir, I think, the contract implies a certain degree of truth. I think you have to be as true to your memory and your experience as you possibly can.