This is so close to what I am going through with my ex. He sent a letter saying that no adoption papers would make him concider our 21 year old his daughter. Also that he never wanted another child so I should take our 5 month old son and live my life. His family has nothing to do with either of the children. His first son from his first wife has never seen his brother for fear of his dad rejecting him too.
The new wife I feel is what has changed him to be a worse father than he was before and he was bad enough before.

Do not even concider him relingusing his rights for then he will not have to pay CS. Even though he chooses not to see his children they deserve the financial help in their lives CS can give even if you put it aside for college etc. Him and his family will one day regret their actions or lack of.

The children will be fine if they have yours and your families love. It will not be easy at times but God will help. There are mentoring programs out there also as they grow you can put them in. The big brother big sister program is but one.

Oh my daughter finally got her dad to talk to her but had to put most of the blame on me so he would call her back. That is a long story that is still playing out in my house today. As for son I have still not heard anything.

Good luck on what you decide but do what is best for the children that you have control over and let God handle the rest. This is what I tell myself every day. I still cry over how he is not seeing his children but not over the divorce.