For this probably apocryphal story to work, it has to be about Zoffany – an artist great enough that Olivier wanted to show off his work, but with a public profile that since his death has never quite matched his amiable talent. For amiable it is: Zoffany has the extraordinary ability to take the rich and powerful, and, while never hiding their wealth, add to it an aura of everyday pleasure, giving the illusion they’re one of us.

Born in 1733, near Frankfurt, the painter originally named Johannes Zauffaly worked for a series of princely courts in Germany, and some of his bland and rather pretty-pretty early work is here. It is only when the young Zoffany arrives in Britain and comes into contact with the actor-producer David Garrick that he finds his own pictorial language.

Zoffany almost single-handedly resurrected the old-fashioned art of the conversation piece – paintings of fashionable people enjoying leisure activities: making music, playing cards, drinking tea, showing off the new possessions provided by the Industrial Revolution.

His early paintings of Garrick are not of the man of the theatre (although those – hugely successful – soon followed), but of the country gentleman Mr Garrick with his wife. Posed by the river, before the Temple to Shakespeare that Garrick had erected in the grounds of his house, Mrs Garrick casually leans her elbow on her husband’s shoulder, even as the formality of the temple is undercut by the delightfully incongruous addition of a pair of boot-scrapers by the entrance. No muddy footprints allowed here.

This homely charm continues even when Zoffany was painting the greatest in the land. In a portrait of Queen Charlotte with her children, the two boys wear fancy-dress, which is a painterly nod toward their future roles, but it is also just a picture of two little boys romping with their mother.

Their father, George III, in another vast image, is dressed in the ordinary redcoat of an officer, slouching informally on the arm of a chair, legs splayed in a display of soldierly masculinity. This could be any landowner, waiting perhaps for his steward to discuss a new land-drainage system.

Despite all the laces, frills and furbelows that Zoffany paints (and he paints them with a highly polished finish that shows how important they were to him and to his sitters), many of his portraits are almost austere in their simplicity. A portrait of Warren Hastings, Governor-General of India, shows him not in full-dress uniform, but in the simple grey-and-brown outfit of a country gentleman. Here, says the painting, is a humble, decent man doing a Herculean job of attempting to reform the notoriously corrupt and rapacious system of crony government in India.

Hastings continues to divide opinion. Was he Zoffany’s melancholic man of integrity, or looter-in-chief? Both positions have their partisans, but Zoffany’s opinion is clear via the simplicity he chooses.

Many of these Indian pictures are revelations. Much of Zoffany’s work on his six-year trip remains to be discovered – several of the pictures here have only been attributed to Zoffany within the last year (one too recently even to have made it into the catalogue).

In India he shows us a world where the Victorian stratification of society into “European” and “native” remain in the future, and images such as The Impey Family show how Indian and British cultures intermixed, as the small Impey daughter, in Indian dress, dances barefoot to the accompaniment of Indian musicians, applauded by her watching father. In other paintings European sitters clutch hookahs, or weave Indian fabrics into their headdresses.

It is always through these objects, through their possessions, that Zoffany’s sitters speak, to their own world and to ours today. At the beginning of the exhibition the visitor is greeted by three self-portraits. In one, hand on his chin, head tilted, the artist stares quizzically out at us, as if wondering how to depict the new objects of manufacture that are now passing in front of him.

For we are the true descendants of his sitters, those people whose rugs, and china, whose teapots, chess-sets and paintings, were so stylishly, lavishly, and enticingly displayed by one of the greatest painters of stuff there has ever been.

The Royal Academy is on something of a roll at the moment. The David Hockney show in the main galleries downstairs is, rightly, one of the season’s hot tickets.

Yet if even a tiny percentage of the Yorkshireman’s fans follow up with a visit to the Sackler Galleries, they will discover a whole new world of grace and charm and light. Follow Laurence Olivier’s advice and be sure to visit the Zoffanys: they love being disturbed.

‘Johann Zoffany: Society Observed’, to Jun 10; www.royalacademy.org.uk

This review also appears in SEVEN magazine, free with the Sunday Telegraph