I don't know how to set a priority, but what I do is I allow it to automatically connect only to the ones I want for normal use (the one at my office, my WAP at home). There are others in the list, but if I want to use one of 'em I'll click on it in the NM display to cause my system to connect to it.
This keeps my preferred access points on auto and everything else I get to choose if I want.