How can I remove only the groups I want from VISIBILITY- CUSTOM USER GROUPS?

You’d have to select CUSTOM USER GROUPS from the VISIBILITY menu and then Add user group one by one.
Is there a way to select “All” and then remove just the groups you want?