Depends what you mean by “wrong.” You might have violated copyright, although on a personal level i don’t think you’ve done anything morally or ethically wrong.
Also, there’s a difference between doing something that violates copyright, on the one hand, and doing something that is likely to get the attention and raise the ire of the copyright owner, on the other.
I would be interested to know, though, what would happen in a case like you describe.
After all, say we take the issue of Fair Use. One of the criteria for Fair Use is the “amount and substantiality of the portion used in relation to the copyrighted work as a whole.” Obviously, copying the whole of a book (as opposed to, say, 3 or 4 pages) is probably going to put you in a whole lot of trouble here. When i was teaching undergraduates, i would sometimes get the library to scan sections of certain books and post them online for the students to read, and the library had very strict rules about how much of a book it was allowed to copy. The reason for this was the “amount and substantiality” section of the Fair Use rules.
On the other hand, though, if you make a copy of your own book merely for your own use and annotation, and do not intend to share it with someone, then it seems to me that you’re not going to run into very much trouble with the other Fair Use criteria.
Your copying seems not to be of a commercial nature (Fair Use issue 1), and it does not seem to have any impact on the “the potential market for, or value of, the copyrighted work” (Fair Use issue 4). Unless, i guess, the copyright owner expects you to buy two copies: one for the shelf, and one to read. I’d be interested to see how all these factors would be weighed in a case like yours.
And so we come full circle to the legal versus ethical question i raised at the beginning. Even if you are in violation of copyright law, in my opinion you’ve done nothing wrong.
I’ve done something similar myself. I have a couple of books that i use frequently, and that i wanted to be able to search quickly and easily. In order to do this, i scanned the books into my computer and ran them through OCR software. Now, i can just open them as a PDF and search for the word that i want. Much easier than thumbing through a 500-page book.
I haven’t shared these books with anyone else, and my only use for the electronic copy is to facilitate my use of a book that i’ve paid for. I don’t know what the outcome would be if i were found out by the copyright holder, but i don’t feel any guilt for doing it, and i don’t feel i’ve deprived anyone of any income from their work.