In Les Planches (Montreux Old Town), just East of the bridge over the Baye river, check for a lane on the mountain side (there are signposts).
With this trail you may go along the river up to Les Avants if you wish. In Les Avants, the MOB train will bring you back to Montreux if you're tired.
Another nice way of discovering this beautiful wild area: from Montreux, go along the lake to Territet, where a steep and impressive funicular will take you up to Glion, the village overlooking Montreux. Walk along the road to Les Avants, you'll see a cemetary on the left, where a trail goes down to the gorge.
Beware: the Gorges du Chauderon trail is an easy high season hike, but don't try it in winter, it would be quite dangerous!