Stop seeing the other man. You are married, he is married.
You need to decide if you want to save your marriage (which seeing the other man won't help) or get a divorce.
If your husband isn't willing to work on the marriage, then there isn't much you can do about it other than choose to live that way or get out. Having an affair will only make matters worse.