There is a small but revealing photograph at the beginning of this show, just behind the door as you come in: an informal snap of Paul Klee and his wife, the pianist Lily Stumpf, taken in 1933. Lily appears to have found something interesting on the end of her shoe; Klee, bent double, is looking upside-down at the camera with a whimsical expression on his face. It is the sort of pose one would expect from someone with Klee's very idiosyncratic outlook on life and art.

The popular image of Klee is that of the solitary dreamer, who "made visible" the world of the imagination, whose spontaneous creativity is epitomised by his famous definition of drawing as "taking a line for a walk". But he was also, as Tate curator Matthew Gale is at pains to point out, "extremely rigorous, with a clear sense of how his work was progressing". From early on in his career, Klee maintained an "oeuvre catalogue" of his own works, giving each a title, a date and a number corresponding to the order of production in a particular year, all of which he would add to the picture mount in his spidery hand.

Very sensibly, the organisers at Tate Modern have adopted a chronological format in the exhibition, using Klee's own ordering system, and this gives it an intelligible structure. There is a selection of works that Klee produced before and during the First World War, but the bulk of the show is devoted to the period 1920–33, when Klee taught art, first at the Bauhaus and later at the Dusseldorf Academy. His output was mostly on a small scale, and the judicious selection of just 132 works spread over 17 rooms certainly gives you the space to get up close, which is really the only way to appreciate them.

What strikes one immediately is the extraordinary variety of Klee's production. The nearest things he ever did to series painting were his so-called "magic squares", works like Redgreen and Violet-Yellow Rhythms (1920) or Static-Dynamic Gradation (1923), which show his abiding interest in colour. Most of the works in this show, like the squares, are essentially abstract, but even paintings that stand alongside each other in sequence, such as Ships in the Dark (1927/143) and Harmony of the Northern Flora (1927/144), are polar opposites in conception. How different, though, they both look to two other works from 1927: Pastorale (Rhythms) resembles a cuneiform tablet (one of the earliest forms of writing) more than anything else, while Seaside Resort in the South of France, appropriately enough, is pure pointillism. For Klee, every work of art was a new departure.

Equally remarkable is the range of Klee's technique. Throughout his career, he was constantly pushing the boundaries of traditional art-historical categories, and experimenting with new media. Many of the drawings on show here were produced by the "oil transfer" method, a sort of homemade tracing process, while some of the paintings employ a spray technique, an effect that he apparently achieved by shaking pigment through an ordinary kitchen sieve. Although a majority of the works in the exhibition are ink or watercolour on paper, no two are alike.

Again and again one is struck by the astonishing fertility of Klee's imagination. Some works, like the satirical Memorial to the Kaiser (1920) or Outbreak of Fear III (1939), have obvious contemporary resonances, but others, the mysterious A Young Lady's Adventure (1922), for example, or the frankly baffling Assyrian Game of 1923, continue to defy interpretation. Surreal touches abound. Exotic River Landscape of 1922 shows a river delta populated with humorously-drawn birds and fish (both recurring motifs in his art). So far, so good. But then, in the bottom right hand corner, you notice a tiny human figure playing a piano. What on earth is he doing there? You are left wondering.

In 1933 Klee was forced out of his teaching position by the Nazis and sought refuge in Switzerland, where he died in 1940 from complications arising from scleroderma, a wasting disease of the skin. By then his reputation as a pioneer of modern art was unassailable; his influence on 20th century art was enormous. Walking round the exhibition, I was constantly picking up clues to borrowings by later artists: Mark Rothko, Bridget Riley, Jasper Johns, Cy Twombly and Keith Haring, to name only a few.

This exhibition is the first to be underwritten by the Tate's new corporate sponsor, EY (Ernst & Young before the rebrand). "We can't be the tail wagging the Tate dog," EY's UK & Ireland commercial managing partner is reported as saying, "but we'll have some choice about which exhibitions we put our name to." They have certainly backed a winner with this one.

Paul Klee, at Tate ModernNicholas Marlowe reviews Paul Klee at Tate Modern.5