It is possible that you may be scared of your partner who is financially strong and is also having a little of negative feedback of he being harsh with others. If this is the situation, then there are chances that he would not have the mental ability to face the situation of divorce. If you suspect any sort of harm to you from him, then you could get the personal protection order from court with the help of the lawyer. Though he was willing to this process in the beginning, later because that his friends are making comments of he not able to control you would change his mind set. Hence you need to protect yourself.