Should I spend time with a friend and her husband if I can't stand him?

A friend of mine is married to a man with the same views of love and marriage as Jim Taggart. He is of no value to me, and I hate being in his presence. My friend invites me to spend time with the two of them and other friends of hers. Should I decline the invitations so that I do not grant her husband any undeserved attention or friendliness? Or should I accept so that I can see my friend? To do the latter feels like insulting my friend and betraying my own values.