I am looking at an identical issue - copying address data over to my AYU_Funds module based on the setting of a Flex_Relate field which could be either an Account or a Contact
See
The address fields in Accounts and Contacts have different names (billing_address_… vs primary_address_…) so the field_to_name_array does not work without some kludge to the setup.
Did your code accommodate the different naming conventions?