By which I mean, why was it given this post? Granted, it is unique as a religious site, I believe, but there must be more to it.
It’s technically a soverign state, and so it makes sense to give it a place in an organization that tries to encompass all nations.
But its not really a state in the way that most people think of it, so it seems a little silly to give it an actual vote.
Hence, “permenant observer”.
And from time to time, countries will call on the Vatican to serve as an arbitrator or go between.
It’s been a long time since the Treaty of Tordesillas, but then again Manuel Noriega liked the Vatican embassy in Panama.
No, the Holy See isn’t a full member because it would be “silly.”
There are a number of competing views on the status of the Holy See which basically boil down to this: the Vatican since 1964 has been basically satisfied with being a non-voting member of the General Assembly, as it means that the Holy See can maintain its status in world affairs (it having diplomatic relations with a large number of countries) and still maintain its role as essentially a mouthpiece for a religious organization without having to be dragged into numerous political, military, and social controversies. Switzerland for a long time held permanent observer status – IIRC they gained that status at the same time as the Holy See and held onto it until sometime fairly recently, the year is escaping me and google isn’t helping me.
Is it the Holy See that’s a permanent observer or the State of Vatican City? 
While the it’s the former that conducts diplomatic relations the country itself is the latter.
A state cannot observe. A representative of the state can. The Holy See provides the latter.
This was news to me, but according to Wikipedia it happened in 2002.
Wikipedia is right! Switzerland officially joined the UN the same day my first son was born, so it’s easy for me to remember.
ETA: for a long time Switzerland didn’t want to join because they were afraid it would jeopardize their famous status of neutrality, and the UN will sometimes take positions censuring certain governments.
IIRC, the U.N. has offered full member status to Vatican City several times, and it has been refused each time, so it’s not as if they’re not allowed to be completely participatory.