IMO, JRDelirious has the right take on it. Castles were, from the Dark Ages through the time of Vauban, constructions that served as strong points for controlling a territory and as defensive structures for the inhabitants in case of invasion.
In the time of Louis XIV, and later in the Victorian era, people built them out of a desire to impress and a sense of nostalgia for the romantic past. So they’re castles by visual impression and by definition, even though they didn’t have the same function.
There were some non-heads of state who built palaces, by the way, though in general they were purposefully the large formal dwelling in which the head of state performed his/her ceremonial duties as head of state. Elizabeth II’s present primary residence, Buckingham Palace, was originally built by the Dukes of Buckingham. Blenheim Palace was the seat of the Dukes of Marlborough. The key point, however, is that palaces are in general not fortified, while castles generally are.
As a note, a castle sensu stricto is not a single interior dwelling, but a castellated wall enclosing an interior area, with at least one fortified tower, the keep, included within it. But typically a castle was a wall enclosing interior open ground and structures.