Optimizing a server is a huge task and it's hard to give a lot of general "do this" recommendations. Most suggestions are specific to an environment.
For starters, I'd make sure that (1) you don't have things installed and running that you don't need. The less PHP modules, Apache/nginx modules and so on that are executing, the better off you'll be. And (2) make sure your configurations for Apache/nginx, PHP and MySQL are as optimal as can be. There are thousands of tutorials online that can help you determine what the optimal configuration for your environment and level of traffic should be, and I'd start with reviewing some of those. Basic things like ensuring static files are cached can go a long ways.