I'm on v4 and I'm trying to figure out if there is a way to set individual blogs as viewable to only private groups or a custom list of members?

Yes, but you'll need to allow users in your groups to do it in the member group settings. 

Go to Admin CP -> Members -> Members -> Groups. Click edit icon (pencil icon) for the group you wish to give the option to -> Select tab "Blog" -> Setting: "Can create private blogs?"