That sounds as good a procedure as any, and one I myself might adhere to.
The unit is proper plug-&-play so there should be no adverse effects from just unplugging it. I would however at least recommend that any applications using it, first be terminated as you describe in your current approach.