Here's the thing, if you move out of state and the courts let you, your children will have extended periods of time ALONE with their father. Unless you can prove that he abuses them or neglects them (and the courts aren't going to take your work for it, they want hard evidence), you won't be able to stop it.
Personally, given what you posted, I wouldn't move. He's less likely to abuse them being an EOW father as he won't have to deal with them for long periods of time. Like he will with a long-distance parenting plan.
And if he goes long periods of not seeing them, then you may be able to move and he may not excercise his parenting time. But there's no guarantee that he won't.
Talk to an attorney to get an idea of what to expect.