Loosely based on the Super Famicom/SNES game Super Mario World 2: Yoshi's Island, Yoshi's Story follows the adventures of six Yoshis as they search through sunny woods, dark caves, soaring mountains, steaming jungles, vast oceans, and gloomy castles for the Super Happy Tree, which has been stolen by the dreaded Baby Bowser. Yoshi can walk, run, swim, leap, flutter, and fire his payload of eggs. The action takes place in storybook "pages" made up of bookmaking materials and other familiar textures, including cut-paper forests, denim clouds, inflatable plastic islands, and stacks of colored spools.

The game pioneers use of the Rumble Pak as a key element of the gameplay experience, the player being able to feel invisible but treacherous currents when Yoshi swims through oceanic depths. The Rumble Pak can also help the player locate hidden fruit or other objects by rumbling when Yoshi smells something. With a light touch of the Control Stick it becomes possible to float on a leaf, ride a friendly dragon, and tiptoe past a swarm of angry bees.