Japanese garden style is just perfect for small gardens. The tiniest space can fit a small Japanese garden. Japanese houses make the most of every bit of garden space, and often have small gardens squeezed a long a narrow pathway, or between two buildings.
A space of just a few feet across is big enough for a tsukubai. This is an arrangement of stone basin, bamboo pipes, and rocks, where visitors can wash their hands before entering the house. Tsukubai are common in small Japanese gardens. If even that's too big, you could just have the stone basin on its own, on a bed of pebbles, perhaps with a bamboo ladle.
Another possibility for a very small space is a deer scarer. This is made up of two pieces of bamboo: one is a spout, and pours water into the other, which is open only at one end, and is fixed in the middle on a pivot. As the second piece of bamboo fills up, its centre of gravity changes, and it tips up and releases the water. When it falls back,it knocks against a piece of wood, and so makes a rhythmic clacking sound.
Slightly larger spaces could fit a dry garden, with boulders and sand or gravel. Japanese gardens often have standing boulders surrounded by mossy ground, but if your area is too dry for moss, you could substitute a low creeping plant such as dwarf thyme for ground cover.
Japanese gardens even make the most of narrow paths, walkways and entrances. You could lay a narrow strip of gravel alongside a path - curve the edge as if it were a riverbank or sea shore - and plant a few shoots of bamboo and a small Japanese stone lantern or other Japanese garden ornament.
For more inspiration, visit Creating Japanese Gardens, or see the photos and videos at the bottom of the page.