i agree that too low is no-go. not that it really harms the car, it just looks crappy. id lower the car like ´BMW does stockwise with their performance packages and M-suspension. max 1 inch. and don´t lower the rear more than the front. when you look at it, the rear is already lower and if you do anything at all you should try to level that. front slightly lower than rear.
against increased tire wear and camber problems you can (and should) have an alignment done. that will give you a proper wheel setup with the lowerance. and, you said your car has 164000 miles on it? you might consider that everything ages...even the chassis of a bimmer. the older the car gets the less it can take really deep and hard shocks. the road bumps will have to be taken by the whole chassis...makes it weaker and weaker you know? just keep it reasonable and you´ll be fine