I’m sure the details of the process are different for the various orders, but in general, the following things might happen.
You could certainly talk to your priest, as likearock suggested, or if you have a good idea of which order interests you, you could approach them directly. Some orders focus on teaching, for example, others are contemplative. Some are cloistered.
After you’ve expressed your interest, most orders that I’m familiar with will encourage you to spend some time with them, to make sure it’s a good fit. This might start with weekend visits. As you move forward with the process, you would join the order formally, and begin a course of spiritual study that would result in the taking of vows. Some orders have various levels of vows, and strongly encourage you to speak up at any point in the process if you change your mind about joining. Obviously, they take the vows very seriously, but at the same time, they realize it is self-defeating to have someone stay if he has a change of heart about being a religious.
On the vow of silence part, most orders have certain times set aside for silence, for prayer and contemplation. From what I know, this varies from an hour a day (the monks who taught us in school) to almost all the time. Also, a monk can take it upon himself to undertake a period of silence. Some cloistered orders have a few monks (or nuns, if we’re talking about them too) who are designated to deal with the outside world … answering the phone, dealing with repairmen, etc, so that the majority of the community can remain in silence and/or contemplation.