That could be because you can't go directly to the Mtn. Wilderness from the Swamp. 
You need to go to the SE Wilderness and once you get to the little pavilion (trust me, you'll know what I mean when you see it) keep following the road along the shoreline. You should end up at what looks like a mine tunnel. Follow that through to get to the Mtn. Wilderness right by the Rapax Rift.
The only other entrances to the Mtn. Wilderness are 1) from the N. Wilderness across a long bridge and 2) if you teleport into Ascension Peak from the Rapax Castle you can walk out of it to get to the Mtn. Wilderness up by Bela.
And yes, 17-20 is more than enough for the Mtn. Wilderness AND the SE Wilderness. You might want to detour past the temple in there for a few nice items after a nasty fight.