Should a person preserve or eradicate his unusual accent?

Here's the original version of the question, which I hope that I've captured well enough with my revisions:

Are accents WHO you are, or something to work to eradicate? Is it rational to feel self conscious about the way you talk (when others seem to understand you)? I'm often embarrassed when a loved one points out my "accent" is showing... I'm a native English speaker by the way, and sometimes catch myself being annoyed by those are aren't - but I'm usually impressed by whatever grasp of a 2nd language people have (when I'm not being annoyed) since my own attempts at learning something else have been less than spectacular. Sometimes I catch myself using a term of phrase or even word that is only connotative to my extended family - to mean what I use it to mean. Half the time it happily reminds me of older relatives / and half the time I wonder if I just sound like a moron.

When I was in my 20s I altered my habitual pronunciations in a fair number of instances. I have this remarkable ability to forget details that I no longer need, such as a locker combination at a gym, when it has been superseded, and accordingly I don't remember the details. In some cases my choices resulted from a pronunciation being less likely to be misunderstood than another; in some cases it appeared that one pronunciation was somewhat more standard than another; and in some it was a matter of subjective taste. None of it had anything to do with getting rid of an "accent". But I think this is an interesting example of the fact that your identity comes in part from your background and is in part self-made. And if someone tells me "If you'd grown up in Wherever, then you'd pronounce this particular word in this particular way", I know that that's not generally true.