Just to throw in my two cents, I like hearing different non-comedy acts. I personally can't stand the comedy songs, although I realize I'm in the minority on that, probably. I usually just ff through them. But a non-comedy performance is good to break up the flow and provide some variety. Space them out and involve them in the rest of the proceedings, and I'm fully on board.
A non-comedy guest, musical or otherwise, doesn't have to be funny, just entertaining. As long as they're willing to join in I give them credit and enjoy listening. Remember, these are some of the quickest wits in the biz so no one expects them to keep up. Just be yourself and play along.