You will have to get served with the divorce complaint and may get called in by the other party as a witness to the case (you may even get deposed). The other party does not have to "catch you in the act" to prove adultery, they just have to show opportunity and inclination. From my experience, most cases where adultery is a grounds for divorce, the complaint gets amended to irreconcilable differences at the time of the divorce settlement.