I've never had a rescue, but my tricolor, Dudley, was a lot like you describe. We got him from a backyard breeder who insisted we take him home at only 6 weeks old. Leaving his mother at that age was clearly detrimental to his development. From then on, he was scared of almost everyone except us, was very shy around other dogs, and had separation anxiety through the roof. We enrolled him in some puppy obedience classes, and he was so stressed out that he started salivating uncontrollably on our first visit. He even started vomiting. We left, and I never attempted obedience classes again. (Maybe this was a mistake? I don't know.) Any time someone would come to the door, Dudley would go crazy--barking, shaking, etc. He was also aggressive and would bite people if he was afraid.
He wasn't the easiest dog in the world, but we managed. He got a little better over time, but he was never the friendly, easygoing dog that my current Cav is. I did the same thing Karlin suggested--didn't allow strangers to touch him (I was afraid Dudley would bite them!), put him in another room in his crate with a blanket over it when we had guests over that he didn't like, and didn't board him. Dud had to travel everywhere with us on vacation because kennels would just send him through the roof. He did eventually make friends with some of my mother-in-law's poodles and my sister's Shih tzu, but we had to socialize him very gradually and at his own pace. He also learned to love some of my friends. They'd come over and sit down on the floor until Dudley decided to approach them. Once he trusted them, they were friends for life.
I wish you the best of luck, and I applaud you for taking in a rescue and trying to respect his temperament. I think that what finally made life with Dudley easier is that I just dropped my expectations that he would ever be a "normal" dog. Just like humans vary in temperament and reflect their early experiences, dogs do, too. He still has a lot of love to give you and your family and is learning to trust you more every day.
Lori