The starting resolution is always 640x480, and has always been that way. The resolutions in preferences indicate the in-game resolutions that myth's 3D stuff will be in - after a game the menus will be 640x480 again.
What do you mean "it does nothing"? Do you mean after you select the new resolution in Preferences, and click "OK" , and then click on "New Game" and select your level and click "Start", that the resolution remains as 640x480? Is that what you're trying to say?