Well, not quite the beginning. There was some controversy when Anselm translated the Bible into the Vulgate (the language of the common unwashed masses, i.e., Latin).
They are still at it, though: they released a 2nd revised Nova Vulgata edition as recently as 1986.
Anselm? The Vulgate was created by Jerome in the 4th century.
There was no issue with translating the Bible into Latin, and many translations had been done in earlier centuries.
Jerome translated it into the ordinary spoken Latin of his time, while earlier translations, besides being bad quality, were in Latin that felt outdated to the people of the 4th century.
According to Wikipedia
The term “Vulgate” is used to designate the Latin Bible only since the 16th century.
My mistake; I don’t know why I said “Anselm”. But the point remains: Although the Church adopted Latin relatively early in its long history, it was not “right from the beginning”.
I get the impression that the early Church fathers were very interested in translating the bible into Greek, because Greek was the language of intellectual discourse, and the Church fathers really wanted to establish that Christianity was intellectually respectable. Latin (though a language of political elites) didn’t have the intellectual cachet.
There was no need to translate the New Testament Bible into Greek, because that is the language it was originally written in.
There was already a Septuagint translation by then, which is consistent with the idea of Greek having some cachet:
“Poētae per litterās hominibus magnam perpetuamque fāmam dare possunt; multī virī, igitur, litterās dē suīs rēbus scrībī cupiunt. Trahimur omnēs studiō laudis et multī glōriā dūcuntur, quae aut in litterīs Graecīs aut Latīnīs invenīrī potest. Quī, autem, videt multum frūctum glōriae in versibus Latīnīs sed nōn in Graecīs, nimium errat, quod litterae Graecae leguntur in omnibus ferē gentibus, sed Latīnae in fīnibus suīs continentur.”
You’re right of course. I should have said that the New Testament was written in Greek (vice Latin) because of Greek’s intellectual cachet.
I’m not sure if you’re joking or not. 
The New Testament was written in Greek because Greek was the most widely spoken language on the eastern side of the Empire. It was written in Greek so that the maximum number of people would be able to understand it.
The early church was most certainly not interested in intellectual cachet, but in reaching ordinary uneducated people, and Greek was the best language for that in the east.
The Roman Empire in the first century AD was to a great extent a bilingual place. Most people spoke a local language as their first language, and then either Latin or Greek (or both ) as a common language – with Latin being the predominant common language in the west and Greek the predominant common language in the east. Greek was the language of government and administration in the east, not Latin.
Sorry, I was the one who introduced the word “cachet” to this discussion; what Cicero actually said in his speech (~62 BC) was “…Graeca leguntur in omnibus fere gentibus, Latina suis finibus, exiguis sane, continentur”, not that (I imagine) he was especially concerned with uneducated people, but the upshot is that at that time (albeit not AD yet) the predominancy of Latin was “exiguous”.
discussions like this is why i love this place …
Don’t you think it’s a bit pretentious to post a passage in Latin without translating? 
This appears to be an exercise from Wheelock’s Latin textbook, adapted and simplified from some passages in Cicero.
The relevant original passage in Cicero’s Pro Archia is
Nam si quis minorem gloriae fructum putat ex Graecis versibus percipi quam ex Latinis, vehementer errat: propterea quod Graeca leguntur in omnibus fere gentibus, Latina suis finibus, exiguis sane, continentur.
My translation:
“For if anyone thinks that there is less renown to be gained from Greek poetry than from Latin, he is greatly mistaken. The reason being that Greek is read in almost all nations, Latin is contained within its own boundaries, which are certainly narrow.”
But note that
a) This was written a century or more before the time of Gospels, and Latin had gained greater ground by then, although Greek was still widespread.
b) He was talking about poetry, not religious texts.
The bulk of the New Testament consists of letters, written by Paul or other important people, to various communities or individuals, and of course were written in the language of those communities or individuals, which was Greek.
Unfortunately, I was serious - just wrong. I think I was misremembering or misinterpreting something I read about Church history about early Christian theological texts, not the New Testament. I’ll bow out before I mess up worse…