Etiquette

I have a question for people in case I'm missing something...  I've been chastised much more than normal lately for passing other riders or not saying "on your left" or something like that.  I've always considered myself to be a somewhat polite and responsible rider so maybe I need to rethink some things?  Here is the situation:  I pass someone with about 3 full feet on the road, on the left, without squeezing anybody or traffic coming from behind or anything, and then I get an earful about not letting them know I'm passing and being an inconsiderate douchebag and the reason motorists hate all cyclists.  This is when I am riding by myself.  I understand that on a group ride the dynamic is a little different.  So the question is, am I out of line?  When other cyclists pass me I could care less.  I don't need an "on your left" or a bell ding or anything.  I'll just ride in a reasonably straight line and if you want to pass, go ahead.  It bugs me when cars honk their horn behind me when overtaking.  Jeez, I had no idea there were other vehicles on the road.  Thanks for letting me know.  I guess I'll just continue riding in a straight line and you can pass?
Its confusing to me when a fellow bicycle rider gets bent out of shape towards another rider on a minor point of etiquette (at least I think its minor) when at least one motorist is going to do something blatantly rude or even downright dangerous during the same ride most of the time.  Something about choosing your battles with fellow road users?  My venom is reserved for a dumb move that requires evasive action on my part to avoid a serious injury.  So far that has been a motorist, every single time.