When I get my own house, it will not be in a town or city. It will be in the country and I will shoot a drone. It would be no different than shooting on my range.
Second flying a drone over someone's property without the owners permission should be criminal trespass IMO. You're not shooting the trespasser, you're shooting a hunk of plastic.
Is shooting a peeping tom okay? No. But, you're not shooting him you're shooting at a hunk of plastic.
I believe drones can cause more harm than good and people will always use them irresponsibly. I have no problem with homeowners using them for target practice.
Better yet, shoot them with less lethal (rubber) buckshot. It will disable the drone, but not "endanger" anyone