There needs to be a lot more information here. How long have you been divorced and waht kind of date? I met my husband's kids at a restaurant once after we'd been dating about 6 months, and then again several months later he re-introduced us and we spent more time with together when he was thinking of asking me to marry him. At that point we went to movies with them, horseback riding, boating. Not the stuff we'd ordinarily do on our dates, but stuff he'd want to do with them, and I came along so they could get to know me.
His ex felt it was all wrong and might have asked the same question you just did, but the kids' therapist at the time saw the progress of things and told my husband that he felt we handled everything just right... about a month after our wedding (planned and executed quickly so the kids would not be put through a long, drawn out process on top of my own parents having health issues that would prevent them from participating in anything that took any longer to put together)... he asked for permission to use some of the situations we'd been through in seminars he did on the subject (including the ex threatening to call police and show up in the middle of the wedding because she thought she'd found a loophole to prevent him from having visitation that weekend, and including a few other nasty little things she did)...
Every situation is different, but remember if the kids are simply coming home and saying, "daddy took me on a date", it may be nothing inappropriate at all... (unless you think it's inappropriate for them to meet his friends and have fun with other adults when they're with him, which might be worth discussing what the limits are on you being able to feel good about him having a new life and them being involved in it).
On the other hand, if you were at the movies when you had your weekend alone, and saw him taking the toddlers into an R-rated movie to sit in the chair beside him and nap while he fondled his girlfriend & necked with her during the flick... well, that's very different and you have every right to be angry and the motivation for taking the kid in that circumstance is simply to avoid having to hire a sitter (or admit to you that he wanted a sitter).
Give us the circumstances and we may be able to actually answer your qeustions more reasonably about what his motives might be.