Is it normal? It depends on whether the application is designed to do the deletion. If it is, and it doesn't delete, then discuss it with the developer.
Should I delete the plugins? Yep - if the application is replacing a plugin, crack on...but maybe talk to the developer to confirm?
One point - the method I use to delete a plugin, provided to me by another developer, works beautifully. And it is a queued task, not an instantaneous task - if you check the plugin listing immediately after the application installation then there is a possibility that the plugin is still there.
So maybe check your background queue task has run...and if the plugins haven't gone then, talk to the developer.