Link Xero contacts to separate client location

Client has multiple locations that is managed under a single group umbrella but is invoiced separately. Currently I can only link Xero contacts to Gorelo at the client level. Would be great to be able to link at the location level as well.