Its something you would require customisation to achieve. There is no built in 'switch off' on that unfortunately.
Sorry, to clarify, you can disable this, but only if your SSO can do admin logins. You just click disable in the admin CP on the same logins page. But if your SSO solution only does the front end, you cant disable it only on the front end