Restrict users to only see peers in their Groups

If we had a global setting called "Restrict users to only see peers in their Groups" which made it so that Group Admins / Users could only see other users in the groups. I think that would solve your problem?

For example

Group A: User 1, User2, User3
Group B: User 4, User3
Group C: User1, User5

User 1 can see: User2, User3, User5
User 2 can see: User1, User3
User 3 can see: User1, User2, User4
User 4 can see: User 3
User 4 can see: User 1