When Manet's picture The Railway (1873) was first exhibited at the Paris Salon of 1874, critics were hostile. The painting depicts a young woman seated in front of iron railings overlooking the sidings at the Gare Saint-Lazare in Paris. She raises her head from the book she's been reading to look up at the viewer with an expression of mild indifference, while a little girl next to her turns her back on us to observe the clouds of steam rising from a passing train. Critics were confused: was this a subject picture or a double portrait?

This is the premise on which the Royal Academy's new exhibition of portraits by Edouard Manet is based: that these narrative genre scenes are in fact portraits – of Manet's family and friends, his milieu of writers, artists, musicians and actors, and important figures from Parisian society. But Manet's portraits are not traditional paintings of subjects in staged, formal poses. An artist who matured at the height of Realism and the coming of age of portrait photographs, Manet's constant preoccupation in his portraits, as Huysmans once stated, was "to envelop his characters with the atmosphere of the world to which they belong". Thus, Manet placed his sitters in natural, realistic situations, turning his subjects into "actors" in scenes of modern life and authenticating his scenes of contemporary life by filling them with real people.

Born into a prosperous upper middle class family, Manet did not need to live by his art, unlike many of his contemporaries, and this gave him the freedom to choose his subjects and approach composition and technique without recourse to convention. He understood what he could take from the Old Masters and his younger Impressionist colleagues, and he used this to forge a new language of visual art with which to depict modern life.

The exhibition, which is organised thematically, begins with pictures of Manet's family. He involved close members of his family as sitters and models in his paintings, and his wife Suzanne is a constant presence in his pictures. She is shown at the piano, in the conservatory, or with a cat nestled in her lap Woman with a Cat (1880). Her illegitimate son Leon Leenhof also makes frequent appearances. In Boy Blowing Bubble (1869) he could have stepped out of an Old Master painting as Manet pays homage to 17th century Dutch and Spanish art. However, in the enigmatic The Luncheon in the Studio (1868), Leon is dramatically forced up against the picture plane in a pose considered daring for its time. Is he leaning on the table, or is he intent on leaving the scene? His eyes look beyond the viewer, suggesting his imminent departure. But behind him are references to classical still life painting: food and crockery set out on the table, the potted plant, the oriental armour are all references drawn from Old Master paintings.

A single room is devoted to Music in the Tuileries Gardens (1862), which is both a group portrait and a painting of modern life. The protagonists in this scene, distributed informally across the entire canvas, are drawn from the artist's social and artistic worlds. Look closely, and there is Baudelaire, Fantin-Latour, Offenbach. And on the extreme left, in a discreet corner, is a self-portrait of the artist himself. The entire painting is rendered a limited palette of soft greys, black, blue and red, all executed with exquisitely articulated brushstrokes.

Meanwhile, in Room 4, Manet's Cultural Circle: Artists, Writers and Figures of the Stage, the writer Emile Zola sits amid the cluttered papers and books of his study and his writerly life, turned slightly away from the viewer, a book open on his knee. Symbolist poet Stéphane Mallarmé, lost in a reverie, lounges in a floral chair, his eyes down, his hand grazing a sheet of paper, with a cigar as a substitute for a pen. Across the room, fellow artist and a regular model for Manet, Berthe Morisot stares hollow-eyed from the canvas. Her grief at the death of her father is portrayed in broad strokes, Manet's favourite black, and deep shadows. Her presence and her sadness is palpable in a portrait that is both powerful and fragile. The end result is one of his most beautiful portraits, a profound expressiveness looks forward to artists like Cezanne and Kokoscha.

The exhibition also features more traditional portraits, including the "status portraits" Manet made of figures such as the politician Georges Clemenceau and journalist, critic and politician Antonin Proust. These are more immediately intelligible to the viewer, portraits intended to communicate to the viewer the power, wealth and position of the sitter.

To help place the works in the context of the city in which they were created, an entire room is devoted to "Manet's World". This display is interesting (including a timeline of his life and photographs from his family albums), but it does feel a little like padding in an exhibition that contains only around 50 paintings. Visitors must wait until the final room of the exhibition to see the outrageous Déjeuner sur l'herbe (1862-63), in a display focusing on one of Manet's models, the striking Victorine Meurent. The Railway, the picture which so dismayed the critics, and Manet's forceful essay in modernism, also forms part of this display, for which Victorine also modelled.

Visitors may be disappointed not to see A Bar at the Folies Bergère (1882), The Balcony (1868) or the infamous Olympia (1863) amongst this display: all these pictures fall into the category of "portrait" as defined by Manet, since nearly all of what he painted was portraiture, but despite these omissions, this is an enjoyable exhibition, offering an intimate glimpse into the lives of his family, friends and colleagues, and the Parisian society in which they existed.

Manet: Portraying Life, at Royal Academy of ArtsFrances Wilson reviews Manet: Portraying Life at the Royal Academy of Arts3