It is my understanding that this is the age where having a father figure in a boy's life is critical to him growing up to be a decent man. Nothing worthwhile is ever easy. If your son is uncomfortable staying with his dad, your son needs to work that out with his dad. Because he's now a teenager, you have the privilege and obligation to stay out of it.
Also, I suggest you stop complaining about the parenting time that your ex doesn't exercise and be thankful, for your son's sake, that he's there for him every weekend.