Perhaps the easy way to comprehend the distinction is to think of Mayor/City Manager. In cities with a Strong-Mayor government, the Mayor heads up the city government. In cities with a Weak-Mayor system, he presides over the City Council but effective power to run the city devolves onto a City Manager employed by the City Council.
In a parliamentary system, power rests in a Cabinet headed by and owing allegiance to a Prime Minister, who must command a majority in the effective house in the legislature to remain in office. He is head of government. In such a system, the Head of State is either a monarch or President who usually combines figurehead embodiment of the national sovereignty with limited emergency powers. Elizabeth II, for example, is constitutionally empowered to resolve a deadlock where nobody has the ability to form a government with a working parliamentary majority. In a couple hundred years, that hasn’t happened but the possibility is there – and it’s her job to resolve it. Same thing for the President of Italy when governments there were unstable, falling in about three months. There’s a famous 1975 case where Australia’s Governor General fired the Prime Minister, for somewhat complex reasons. A Prime Minister serves at the pleasure of the “main” house of parliament, in Britain’s case the Commons, and they can vote him out of office by a no-confidence vote. In some parliamentary governments without a strong two-party system, this led to short-lived coalition governments. Germany has a policy in place that a vote of confidence can only be held if there is a proposed alternative coalition organized to replace the incumbent government if it is defeated.
In a Presidential system, such as the U.S. has, the jobs of Head of State and Head of Government are united in one man, the President. There are advantages and disadvantages to this system vis-à-vis the Parliamentary one.
France is an odd case, mixing authority, so they have both an active President and an active Prime Minister, both with a large measure of power to run the country. This solution was designed by Charles DeGaulle in 1958 to end the problem of successive coalition governments falling apart in short order after being formed. AFAIK, the French system has not been imitated elsewhere, but it works for them.