Their not going to give over physical custody of an infant to the father if you are not an unfit mother.
Joint custody is more of a legal term giving both mother & father say in medical issues. Most people work it to be joint custody with physical/custodial custody to the mother & visitation for the father. In this case the non-custodial parent (this scenario the father) would have to pay the custodial parent child support which is based on both incomes and usually a state guideline formula & then childcare is figured in if it's needed.
Save yourself some effort, strength & frustration by trying not to keep having these yelling arguments with him. He's a louse, even if he knows it he's going to deny it.