The father has parental rights. You can't deny him reasonable access to his children. Defining reasonable is a trying legal exercise (a.k.a. custody battle).

What you can legally do RIGHT NOW is deny him access to yourself. Don't invite him to your home until he cuts ties with the crazy girl. If he can have both, he'll take both.