Is listening to music from YouTube a violation of intellectual property rights?

Given that content creators can remove YouTube videos that violate their intellectual property rights, is it wrong to assume that they consent to the posting if they've not asked to remove it? It is wrong to watch or share clips that seem to be uploaded without permission? It is wrong to download music from YouTube for my own personal use, whether uploaded by the creator or someone else?