I just checked my live site which was and an upgrade from 3.x, and all the tables are utf8.
I just checked my development site as it was a fresh install, and there was a mixture of utf8, and utf8mb4.
All tables that start with a ipf_ are utf8, and all the others are utf8mb4. Shouldn't all the tables be utf8mb4 ?