I would not be too concerned about the play date you had. I think Karlin is correct about the age difference being a good part of the issue. When we have our 2 litters of puppies (now 6 weeks) out of their pen with the adults, the adults that are most interested (other than the 2 moms) are the younger ones that are in the 9 to 12 month old range. They play together until everyone is exhausted. On the other hand, the older adults play for a short period of time and then the puppies become "pesky little critters" and the adults look for higher ground (the couch) where the puppies can't get at them so easily.
On the other hand, we also dogsit for other Cavaliers. We have found that it takes anywhere from half a day to 1 1/2 days for the adult visitor and our dogs to adjust. During this adjustment period we have to keep a close eye on them while everyone gets used to each other. We occaisionally see signs of stress during the "break-in period", but it usually doesn't last. We also tend to crate the visitor every so often for an hour or so to give him a break from our crew; our 9 dogs can be a bit overwhelming at times.
Hope this helps.