What you describe will work, though unless you have a need for individual permissions, it's most easily done on a per-Group basis rather than a per-Member basis.
(And yes, with this method you will be granting them specific permissions to the ACP, not the ACP as a whole.)
Create a new Group, named however you wish, then grant that Group restricted ACP permissions on the ACP -> Members -> Staff, Administrators page.
Everyone in that Group would have the permissions you grant.
You can do it the other way, on an individual member account basis, but that's more administrative work.