If my ex-wife and I agree to permanent spousal support, do I have to pay even if she remarries?

No, permanent alimony is not permanent in that sense. Certain conditions will cause permanent spousal support to cease. Remarriage is typically one of them.