Would be useful if you post video with debug printbolds of normal + calculated angle.

And show your exact code with your orientToNormal function.

And try what IzNoGoD said: portal.angles = vectorToAngles(trace["normal"]);

And add more surfaces to your map, which aren't all straight walls. Lots of different slopes to test all kinds of trace normals (or just test in a stock map, should have more variety in surfaces).