Okay, we have like four different sounds to discuss at the moment. The first is /p/, which is an unvoiced, unaspirated bilabial stop. This is the sound of <p> in English spit, and it’s the sound of <p> in Classical Latin as well. If you speak Spanish, French, or Italian, the sound should be familiar from those languages.
However, unvoiced stops in English are aspirated most of the time - that is, they’re pronounced with a puff of air after them (you can feel it if you hold your hand in front of your mouth. Contrast spit and pit, and you’ll feel that puff of air right after the /p/ sound in pit). This sound is written phonetically as /p[sup]h[/sup]/. Unvoiced stops at the beginnings of syllables in English are aspirated. Thus, pit is pronounced /p[sup]h[/sup]It/ (with capital <I> standing in for that particular vowel in the particular transcription I’m using), tit is pronounced /t[sup]h[/sup]It/, and kit is pronounced /k[sup]h[/sup]It/. Whereas the corresponding sounds in Latin, those same /p, t, k/ sounds, are never aspirated - that puff of air is not released.
So we’ve talked about /p/ and /p[sup]h[/sup]/. Sound three is the one Giles mentioned - it’s an unvoiced bilabial fricative. It sounds a lot like /f/, but it doesn’t involve the front teeth. You pronounce it by creating a narrow opening between your lips and creating friction there. It’s written in SAMPA transcription (which I’ve been using thus far) as /p/ - p-backslash. (The forward slashes note simply that the enclosed text is a phonetic transcription. Can I say how much clearer this would be if I weren’t limited to normal characters? :))
Sound four is /f/: that’s the sound we English speakers use in Greek-derived words like philosophy and perfectly native words like friend. It’s a labiodental fricative: a fricative pronounced with the top teeth and the bottom lip creating the friction in the airstream.
Notice one thing: all these sounds are made with very nearby parts of the mouth, and they’re all unvoiced. It’s not surprising that they have gotten shuffled around at various points in language evolution.
Okay, now in Classical Latin times, most well-educated people studied Greek, and spoke it to some degree. Classical Latin, as I said, used the letter <P> to represent /p/; there was to /p[sup]h[/sup]/ sound. But Greek, as spoken at the time, had a /p[sup]h[/sup]/ sound, which they wrote with the letter phi. Notice something: the sound the Greeks used in the word philosophia was exactly the same “p-sound” that we use in peal or pill.
To the Romans, it was thus perfectly logical to transcribe that sound as <PH>, since it quite literally was a /p/ followed by an /h/. Greek words borrowed into Latin were pronounced, at least by educated Romans, according to the proper Greek pronunciation (much like some English speakers make an effort to pronounce words borrowed from French according to French phonology.)
As such words filtered into Latin and began to be used by people who weren’t educated in Greek, however, folks started pronouncing them not with the foreign pronunciation but with the closest they could come in their own language. And that turned out to be with an /f/ sound. I couldn’t tell you exactly when this change happened; no doubt it was a gradual one and it might not have fully occurred until the early development of the Romance languages. The important thing, though, is that they were pronouncing foreign words using the closest sounds available in their native language.
Now, as has been pointed out, such words were spelled phonetically in some Romance languages. Spanish and Portuguese, at least, spell such words with an <F>. But French spelling has always tended to be a bit less phonetic and to preserve historical spellings (in fact, in recent centuries, silent letters have been added back to French spellings to come closer to Latin spelling!) When we inherited such words from French, we got the French spellings and pronunciations. Thus, the French philosophie came complete with <PH> and with the /f/ sound. And this has been preserved in English (which also has a history of being rather unphonetic, so it’s not really much of a shock that we kept these odd spellings.)
Confusion between those four sounds is actually pretty common in borrowing across languages. Take the language known in English as either Farsi or Persian: the /f/ sound is present in the native pronunciation, but when it was borrowed into Arabic, which lacked the sound, it was instead pronounced with a /p/. The Arabic word, in turn, got borrowed (no doubt through a chain of intermediaries) into English, and it’s only more recently that the word Farsi has been borrowed back into English. Similarly, the word “Filipino” is pronounced “Pilipino” in Tagalog, the most widely-spoken language of the Philippines.