Do you remember that there was a time in Windows when all the UIs used to have the three magic buttons: 'Ok', 'Apply' and 'Cancel'?
- Ok would apply the settings, and close the dialog,
- Apply would apply the settings, and leave the dialog open and
- Cancel would close the dialog.
My brother used to tell me, man, it's easy to implement this, since 'Ok' becomes only:
And here it is (almost) the same reminiscence in Eclipse (org.eclipse.jface.preference.PreferencePage).
Oh man, how times have changed... oh wait.
PS. If you want to apply the changes, and keep the dialog open, because you want your users to be able to preview the changes, just make those buttons: 'Apply' and 'Close'.