The important thing isn't the house. It's the ability to make it. You carry that in your brains and in your hands, wherever you go... It's one thing to carry your life wherever you go. Another thing to always go looking for it somewhere else.
If one wants another only for some self-satisfaction, usually in the form of sensual pleasure, that wrong desire takes the form of lust rather than love.