Giving fans what they don't necessarily want

Bands have a dilemma: balancing the desire to play new material at concerts against audience expectations of their greatest hits.

Ben Goldwasser, left, and Andrew VanWyngarden of the band MGMT perform at the Coachella Valley Music and Arts Festival in Indio, California.

Picture the scene. You're at a gig you've looked forward to for ages. The hits from the band in question have soundtracked your every move for the past six months. This will be an evening where you can let yourself go and sing along to your favourite songs at top volume. And then the frontman leans into the mic and tells you: "This one's a new one." It's one of the most deflating sentences in the rock lexicon, but shouldn't it actually be one of the most encouraging?

Such brave decisions to give the crowd what the artists think they need are becoming more and more prevalent. Take Tom Jones, who's just played to a packed crowd at the Latitude festival in England. His audience - surprisingly diverse in age and tastes - had come prepared to hear songs from his excellent new album Praise & Blame, but also had every right to expect they would sing along to Sex Bomb, Delilah or It's Not Unusual. Instead, Jones ambled his way through Praise & Blame in its entirety, thanked everyone profusely for staying with him, and left. For a man well versed in giving his fans exactly what they want, it was probably a rather odd experience for him, too.

In the end, Jones played a great set. But he's not been alone this year in surprising an expectant throng. Earlier this month, Lou Reed played to a chorus of boos as he "treated" his audience to, as one member put it to the Montreal Gazette, "discordant noise lacking melody, style, beauty or skill". There was certainly no Walk On The Wild Side, despite that classic song from his Velvet Underground days earning an explicit mention in the programme.

True, there was a clue that this would be no ordinary show. Reed was playing the Montreal International Jazz Festival. But it didn't prevent people from walking out after his bandmate John Zorn invited those who had paid nearly U$100 and didn't think they were listening to "music" to do just that. In the end, surprisingly, concertgoers who did leave and asked for a refund actually got their money back.

It's not just the "heritage acts" who play fast and loose with their own back catalogues. MGMT's return to touring earlier this year was remarkable not just for the fact that the new record they were promoting, Congratulations, contains absolutely no hit singles whatsoever. They also refused to play Kids, their most recognisable song to date. All of which was perhaps understandable - MGMT wanted to concentrate on their new material - but in the end remarkably arrogant. It's a five-minute song everybody in the crowd would have wanted to hear. Could they really not have spared the time to play it?

At the same time, though, such determination to confront audience expectations should be applauded. If you're about to see a big band, such as Coldplay or U2, playing live, going on the fan forums beforehand is like reading a spoiler for a forthcoming film. The set lists for these groups are immoveable beasts - thanks to the tightly choreographed pyrotechnics, no doubt - which remain unchanged from city to city. It means it's easy for arena bands simply to go through the motions.

So there's a genuinely wide chasm between purely entertaining an audience and making a live artistic statement. It's not a dilemma confined to music, either. The comedian Rhod Gilbert often bemoans the inherent paradox in stand-up comedy. If he creates a set of all-new material, then he runs the risk of disappointing fans who, often, have learnt his best jokes via the best-selling DVD and want to hear them retold, live. But if he does choose to include some of his better work collated over the years, he's accused of re-hashing the same old jokes.

Surely, though, we want our favourite bands and comedians to keep developing creatively. It's what keeps them fresh, vital, interesting. It's just that it appears we'd prefer them not to do so in the live arena - where somehow they're shortchanging the fans by not playing a crowd-pleasing set. So is there a nice, utopian middle ground where the hits intermingle with promise of some (good) new material? Yes. It's those tours where a band plays an album in its entirety and then some other fan favourites in the encore. You know exactly what you're getting with them. But who ever wanted rock'n'roll to be that boringly prescriptive?