What exactly does the decree state? Does it just say that you have to ask your ex if he wants to watch your daughter and vice versa if you have an appointment etc.?
If nothing is specifically stated just that you have to ask each other then I would tell him that he is to drop her off at your house and he can pick her up the following morning. Period.