Its a double wall pipe. The back of the instructions would have some drawings of it.
You would have to take a picture.
Usually if the pipe is as wrong as you make it sound the fireplace would not even run.
Didn't you already try the bag trick, taping a bag over the term cap to see if it would help, and it did not? Taking it a step further, while you have the cap off you can jam some rags in then vent when its cold outside and see if that stops the air coming in. If it does not stop, you have just eliminated the vent (which I think we did once already).