You asked for it...
Some polymers display a nonlinear elastic behaviour. Meaning that on the stress strain curve it does not 'unload' along the same path as it was 'loaded'. The loading curve is typically sharper or steeper and therefore has a higher stiffness (requires more force to displace or stretch it) and the unloading curve is typically shallower, therefore having a lower stiffness (requires less force to displace or shrink it).
There are some that will exhibit perfect linear elastic behaviour (meaning they load and unload along the same line) and will be the same in either direction. So I should have said that for some kinds of elastics pulling it is harder than letting it pull.