Why is self-interest moral?

I was raised to think that morality is a restraint on self-interest, so that we don't harm others by pursuing what we want. However, you seem to think that morality is nothing more than self-interest. Why? Do you think that it's moral to hurt others to get whatever you want?