I have a German project. The date is shown like:

September 20, 2021

It should be:

20. September, 2021

Where can I change this? I have Plesk with Debian (both latest). The Locale for language in ACP is correct.

This is not crucial, but I do not like this. Can I change it somewhere? Is it OS, Plesk or PHP setting? 

Depending on the location you are seeing those, it may not be a locale setting you are looking for. Take a look at the guide here