Life with such people is not impossible, but extremely difficult. Loneliness is the worst thing. You are with him and yet you're completely alone.

As for what to do - it's only for you to decide.