Sure, indentured servitude could be terrible. There’s no doubt about it. In fact, one of the points often made by historians about labor in the British North American colonies during the 17th century was that the lives of slaves and the lives of indentured servants were often not that much different from one another. This was particularly the case in plantation societies like Virginia.
If you read the records of the Virginia Company, and some of the court documents that are available from 17th-century Virginia, there are numerous instances of indentured servants going to court to protest against brutal treatment at the hands of their masters or mistresses. Beatings and whippings, withholding of food, incredibly long work hours were some of the complaints.
Because the magistrates in these courts tended to be wealthy colonists, the type who might have indentured servants of their own, and who would be more likely to identify with the masters and mistresses than with the servant, this meant that the servants bringing these cases often had their claims for mistreatment rejected. In some cases, indentured servitude also came to be more and more like slavery, because masters and mistresses could, and did, take servants to court for alleged laziness or failure to fulfill the contract, and sometimes the magistrate would tack extra years onto the end of the indenture, beyond the originally-agreed period of time.
Of course, a couple of differences with slavery emerge in stories like this. First, however badly they were treated, indentured servants had, at least, signed their indenture contracts voluntarily. This doesn’t excuse, and nor should it minimize, the terrible treatment they received, and they often signed contracts unaware of how hard their lives were going to be, but at least they weren’t simply kidnapped and sold to a slave trader; they made the decision themselves about whether to accept the loan, and whether the length of the indenture was an acceptable risk for them.
Second, as English men and women, indentured servants had rights, and they did have access to the court system as a potential remedy when those rights were violated. And even though magistrates tended to empathize more with masters and mistresses than with servants, some indentured servants did actually have their rights vindicated in court, with remedies that included reductions in service length, additional end-of-service payouts, and other forms of compensation.
Your last point about the connections between slavery and race is important here. One historical interpretation of the dramatic rise in slavery in Virginia in the late 17th century is that those who ran the colony were increasingly worried about class rebellion among indentured servants and landless former servants, as happened in the Bacon’s Rebellion uprising of 1676. While the rebellion was quashed, leaders in Virginia sought to prevent the resurgence of class hostility among English colonists, and in order to do this they began to replace indentured servitude with slavery, and to write laws that more and more explicitly defined slavery in racial terms, and that regulated the lives of blacks more and more strictly, creating a racial divide that replaced the class divide as the most important social and political and civic distinction in Virginia. They added anti-miscegenation laws, outlawing interracial relationships, which had been somewhat common among poor white and black indentured servants and slaves. Poor whites were encouraged to identify with wealthy whites, and to see themselves as free Englishmen in comparison to the enslaved blacks in their midst.
This interpretation of Bacon’s Rebellion, and of the efforts to create a race-based rather than a class-based society in Virginia, is most fully spelled out in Edmund Morgan’s incredibly influential book American Slavery, American Freedom: The Ordeal of Colonial Virginia, which is required reading for anyone seeking to understand Virginia in the 17th and early 18th centuries. Historians debate over Morgan’s interpretation, and there are disputes as to how important this concern about class conflict was in contributing to the shift to slavery, but almost all historians agree that it was, at least, one factor. There were, of course, other reasons for the shift to slavery in Virginia, including the expansion of the colony and tobacco growing, leading to a need for more and more labor; the increasing health and decreasing mortality rates in the colony, which made slavery increasingly cost effective, compared to indentured servitude; the improving economic conditions in England, which reduced the number of poor English men and women willing to sign indenture contracts and migrate to Virginia; and the growing British control over the Atlantic slave trade, with Britain the most powerful slave-trading nation by 1700, which made slaves more readily available for purchase.