Thank you for the musical

As High School Musical bounds on to the stage in Dubai and Mamma Mia! sashays its way across the big screen, Iain Hollingshead sings the praises of an enduringly popular but often-maligned art form.

The cast takes a curtain call at last year's London premiere of High School Musical Live on Stage at the Hammersmith Apollo.

Most men, if pushed, will admit to possessing a guilty secret. A collection of Take That records under their bed, perhaps. Or a cigarette habit they hide from their wives. Mine? Well, it is far, far worse. My name is Iain Hollingshead and I quite like musicals. Last year, I even co-wrote and produced one: a political satire called Blair on Broadway, which started in a fringe theatre in north London and briefly made it to the West End.

In these enlightened times, when men are allowed to bring up children, cry in public and even secretly use moisturiser, admitting a predilection for musical theatre is still viewed with deep suspicion. And yet I cannot be alone. Despite an economy in the UK that is widely thought to be going to mush, presided over by a prime minister few like and no one elected, it was announced this month that London's West End broke all records in 2007. "The tills are alive with the sound of music," sniggered the ­punning headline writers.

Attendances were up to 13.6 million, 10 per cent higher than in 2006. Box-office takings rose by 18 per cent to almost £470 million (Dh3.5 billion). Sixty-five per cent of West End theatregoers watched a musical, compared with 22 per cent attending straight plays. Not all of them could have been hen parties or busloads of menopausal ladies from Surrey, holding their breath until the cathartic release of the famous line near the end of Dirty Dancing, "No one puts Baby in the corner".

In Dubai, meanwhile, the stage production of Disney's international box-office hit High School Musical is making its Middle East debut with a four-day showing until Saturday. Last week also saw the release of the film version of Mamma Mia!, the hugely successful stage musical of Abba songs. Starring Meryl Streep, as well as the incongruous pairing of Pierce Brosnan and ­Colin Firth - an all-singing, all-dancing James Bond and Mr Darcy - it has opened to mixed reviews; some critics have suggested the erstwhile Bond holds a martini rather better than he can hold a tune.

If the film is as successful as the stage version, it will be a licence to print money, money, money. The stage show Mamma Mia! has been seen by 30 million people worldwide since it opened nine years ago and currently boasts 18 productions in 10 countries. It has taken more than £1 billion (Dh7.4 billion) at the box office at a current rate of £4 million (Dh29.4 million) per week. Musicals are obviously huge, if risky, business. And yet there is no comprehensive answer as to which magic ingredients distinguish a success from a flop, a long-running hit from a short-lived turkey.

"Nobody knows what will work and what won't in musical theatre," says Andrew Lloyd Webber, the almost unimaginably wealthy composer of hits such as Cats, The Phantom of the Opera, Evita, Jesus Christ Superstar and Joseph and the Amazing Technicolour Dreamcoat. "The simple fact is that, if we knew what the formula was, there wouldn't be any unsuccessful ones," says Michael Ball, who played Marius in the original production of Les Misérables and is currently starring in Hairspray.

Certainly, critical success or ­failure is not sufficient alone to make or break a show. When the comedian and author Ben Elton collaborated with Lloyd Webber on Beautiful Game, a political musical about football set in Northern Ireland, it received rave reviews but closed ­after only a year, losing £2.5 million (Dh18.3 million). Elton went on to write We Will Rock You, an ­execrable jukebox musical of Queen's songs that the critics panned. Six years ­later, it is still going strong in ­London's Dominion Theatre, mainly attended, one reviewer suggested, by lonely eastern European plumbers keen for a karaoke singalong.

According to Lloyd Evans, The Spectator's theatre critic who has also written musicals himself, critics don't really "do" musicals. "They are superficial, which is part of their appeal," he says. "But that doesn't give us critics much scope to do our 'I'm a deep and meaningful commentator' thing. Instead, we just have to say 'Fantabulous!' or 'Supersensational!' or something equally embarrassing. We're literary people; we can do story, character, suffering and all that, but not a tune and a gavotte."

Neither do a big name or a big budget guarantee a success. The musical version of The Lord of the Rings, a successful film franchise, was the most expensive theatre production ever with a budget of £12.5 ­million (Dh92 million). And yet The Daily Telegraph, aping the philosopher Thomas Hobbes, accurately predicted its run would be "nasty, brutish and short". Although seen by almost half a million people, it closed on July 19 after just over a year, having suffered a massive loss.

An even bigger turkey was the musical adaptation of Gone With the Wind, which closed in June after 79 performances. "Connoisseurs of big, bad musicals must rush to catch it in case it's quickly blown away on gales of ridicule," wrote the Evening Standard theatre critic Nicholas de Jongh. It flopped despite being directed by Sir Trevor Nunn, one of the most bankable names in British theatre, and taking its story from one of the most popular films of all time. Margaret Mitchell's novel that inspired the film still makes around £500,000 (Dh 3.7 million) a year in sales for her estate.

Other film-stage, stage-film crossovers have, of course, been more successful. West Side Story, The Sound of Music, The Lion King, Dirty Dancing, Chicago, Billy ­Elliot and, most recently, High School Musical have all triumphed in both mediums. High School Musical, an endearingly naive feel-good story, was first shown on American television in January 2006, and became the Disney Channel's most successful home-grown movie, with almost eight ­million people tuning in. The London stage version sold out almost immediately after a successful nationwide tour. Internationally, it looks set to be equally successful in its stage incarnation, with huge excitement over its seven performances at the ­Centrepoint Theatre in Dubai. The all-­Dubai cast of 90 was chosen from almost 1,000 applicants.

Other musicals have ping-ponged even more complicatedly backwards and forwards. Hairspray started as a musical film, became a musical and was then remade last year as a musical film starring John ­Travolta. My Fair Lady, which has been successful both on stage and celluloid, is rumoured to become a film remake starring Keira Knightley and Daniel Day-Lewis. Most confusingly of all, The Producers, a story about an attempt to stage a flop that became a hit, was a hit in its stage and first film incarnations, but a flop on its film remake.

Clearly, the two-way path between the stage and the screen rarely runs smooth. Crucially, a film version of a musical lacks the theatre's fourth wall; it can't rely on the audience to provide its energy. The producers of Sweeney Todd, the slick film adaptation of Stephen Sondheim's original, starring Johnny Depp, seemed reluctant even to tell their audiences that the film contained songs. Complaints were made to the UK ­Advertising Standards Authority that the trailer didn't adequately label the film as a musical. Some audiences were even reported to have walked out in a huff.

If it was a deliberate obfuscation, it reveals something of the British public's ­approach to musicals. There is undoubtedly a degree of snobbery, some of it more multilayered than might at first seem apparent. One group - the true snobs - would sooner be seen dead that caught attending shows they think only suitable for gullible tourists and slightly ghastly people from the suburbs. Another, marginally less snobby, group might make concessions for Les ­Misérables, which is more opera than musical and is, after all, based on a very long novel by a French person, or for Avenue Q, which is very witty, or Spamalot, which was written by the Monty Python team who did, at least, go to Oxbridge. A third group - the musical purist snobs - will only watch anything by Sondheim or, at a push, a revival of the ­golden age of Rodgers and Hammerstein. And then there is everyone else, happily queuing to pay £40 (Dh294) for a night of high-kicking spandex.

As someone whose own attitude towards musicals has shifted from somewhere around the second group to nearer the fourth - my Damascene conversion came when forced to attend a three-day Mamma Mia! acting course for another newspaper and thoroughly enjoying myself - I have no more insight than the likes of Lloyd Webber into what makes a successful musical, but it is, perhaps, more instructive to consider why the genre continues to enjoy such success.

Cynics point to the boost provided by the popular BBC television talent shows that have recently cast for West End productions of Oliver!, The Sound of Music and Joseph and the Amazing Technicolour Dreamcoat. Kevin Spacey, the Hollywood actor and ­artistic director of the rather serious Old Vic, complained that this amounted to free advertising at the expense of their rivals. Spacey makes a fair point. But the complaint about the saturation of supposedly unchallenging musicals is nothing new. In 2000, Sondheim said that Broadway had merely become "a tourist attraction". Back in 1954, The Observer's theatre critic complained that 27 of the West End's theatres were offering light comedies and musical shows. And yet neither criticism adequately answers the essential argument that the current crop of musicals supplies the public's seemingly unquenchable demand.

A significant part of this demand is met by the unique ability of music in general - and musicals in particular - to carry the big emotional moments that straight ­drama cannot. As Richard Eyre, the director of Mary Poppins, put it: "People on stage burst into song when mere words are no longer sufficient." The director Nunn has spoken about music "giving the individual story an epic dimension". The desire to clap regularly during a musical is often involuntary as well as cathartic.

More prosaically, even a bad musical has something to divert you. "If the story sags, enjoy the tunes," says The Spectator's Evans. "If the tunes are dull, watch the dancing." Many of the best storylines hark back to an innocent, more wholesome age. ­Mamma Mia!, for example, is the upbeat tale of a young bride-to-be seeking out her real ­father on the eve of her Greek island wedding. Explaining his decision to appear in the film, Brosnan said that he had seen the stage show in London and "found it so wonderfully happy and joyful that I said yes to the movie offer right away".

Other shows draw successfully on well-known, almost collective, narratives that are already in the public domain. Cats owes its lyrics to TS Eliot poems. My Fair Lady is George Bernard Shaw's Pygmalion with songs. West Side Story and High School ­Musical are both essentially rewrites of Romeo and Juliet, although High School Musical has a somewhat happier ending. Reviewing the stage version, which opened recently in London, The Daily Telegraph's Charles Spencer wrote, "At at time when so many of our teenagers appear to be walking the streets armed with kitchen knives, the show's vision of youthful hope, innocence and love seems especially appealing."

This comfort blanket notion also gives rise to a convincing theory that musicals do better during times of political and economic strife. Three of the best film musicals - The Wizard of Oz, Meet me in St Louis and ­Yankee Doodle Dandy - were released during the Second World War. A further six of the American Film Institute's top 20 musicals of all time were released during the Vietnam War. To date, the West End's musicals have kept ticking over while the rest of the economy slows and the world gradually becomes more dangerous again. People always need to laugh, especially in extremis.

Musical actors certainly recognise this. "Appearing in a musical is like being hooked on a drug," says Joshua Martin, who brilliantly interpreted the former prime Minister in Blair on Broadway. "As soon as I arrive in the wings and the overture starts, all the worries of daily life are left behind. I'm transported to another, usually happier, place and it's my job to take the audience along with me."

As long as this enthusiasm rubs off on the audience, it is difficult to leave a musical, even if it is We Will Rock You, without smiling. After all, when faced with the humdrum reality of life at the office, rising food prices and runaway fuel costs, what more could you possibly want from a night out?