Its very very likely this is related to email. If there are only a few followers, it would send out the emails right away. So if there is an issue with that email sending (the server taking too long to respond), it can delay that process. It having saved in the database, would actually point to this even more so.