Suppose I know the FBI or whatever gov agency is monitoring me, and I find eavesdropping devices in my home. If I destroy or remove them have I committed a crime?
I doubt it. You could always claim in defense that you didnt know whose bugs they were or what they were.
I recall a story of an FBI guy who was sent to the Den of Mafia Thieves to recover the listening devices the FBI had installed in the ceiling of the restaurant. The Mobsters gave it to him. Professional courtesy I suppose.
This is not legal advice. I’m not your lawyer.
You could remove them. I know of no law that requires you to leave even court-ordered bugs in place after discovering them on your property. I suppose you could be charged with obstruction of justice, but I highly doubt it. When the mayor of Philadelphia discovered bugs (widely presumed to be placed by the FBI, which was investigating municipal corruption) in his office a few years ago, he removed them and suffered no consequences for it, IIRC.
Practically speaking, of course, unless the FBI was done with their surveillance of you, you could expect the bugs to be put back in - and even better-hidden - next time. Probably better to leave them there, and be very careful what you said around them (and be sure to loudly praise the highly professional, hard-working men and women of today’s FBI). 
Do you have a bug hunting license? 
I would suspect that unless you are a professional in such matters you are not likely to find any FBI or similar agency bugs anywhere anytime.