Removing Subpanels from Certain Security Groups\Roles

Hi there,
I am trying to remove some subpanels from a custom module, however I only need for them to be removed for certain Security Group\Roles.
Within the security role i have, I tried setting the modules I want hidden to disabled and list to none but this doesn’t seem to have worked, is this a bug or have i missed something out?