Depending on what you consider “bone”, the oldest bones are either conodont elements or ostracoderm armour. We have actual examples of both that retain the original mineralization. Around 480-450 Ma is a good timeframe.
I’d say conodonts, myself, they seem to be around around 10 Ma before the armoured fishies. The elements are hydroxyapatite, the same mineral that makes up much of bone.