The Arkenstone of Thrain was found by the Dwarves under the roots of the Lonely Mountain. The Simarils were made by the great Elven smith, Feanor, in the land of Valinor, beyond the Western Ocean.
There were three Silmarils. One was lost in a volcano, one was thrown into the sea, and one is now the morning star. (Or is it the evening star? Tolkien was a little vague.) They have been scattered to the earth, the sea, and the sky, and they shall not be together again until the world is unmade.
You could argue that the one in the earth could have ended up in the roots of the mountain. However, Tolkien describes the Arkenstone differently than he describes the Silmarils. The Arkenstone "shone like silver in the firelight, like water in the sun, like snow under the stars, like rain upon the Moon!" The Silmarils glowed with an inner light: the light of the Two Trees of Valinor, of which the Sun and the Moon are pale reflections.