His friends and his mother were a big issue in our relationship