Could an individual state establish itself as a monarchy and still be in the union?

True, but having a head of state who gets the job simply by virtue of birth seems the antithesis of a “republic”.

Although it is true for many of the monarchies that are currently around that the monarch gets the job by virtue of birth, this is not a requirement for a monarchy. In history there are examples where a monarch is elected by the nobility (e.g. King of Germany) and could even be recalled under some circumstances.
Of course, as long as nobility as a whole is out of the question, you cannot have a “king”, but the differences between a king as described above and a Governor for lifetime are pretty small.
Far fetched as it may be, a state may be ok with calling itself a “Monarchy” as long as it’s only a title and the people have enough control over the government (i.e. “republican” form of government).