Change Security Groups Access

I have a client that imported all their users into CRM when that was done everyone had the same access to all security groups. Is there a way to remove users from a group but leave the correct user in place? I am wondering if I can do it in sql but not sure if it is worth the time. I have 24 groups that need to be cleaned up. Each group has 24 users in each so I would have to manually remove 23 user. Let me know what might work best. Thanks