MIGRATE VMS FROM VMC TO ON-PREM WHILE KEEPING NSX SECURITY TAGS

Share on:

VMC NSX tags

In this post, we’re going to see how to script the migration of some workloads from VMC to a datacenter on-prem, leveraging HCX for a live migration.

At the time of this writing, HCX doesn’t copy the NSX tags when doing a VM migration. The following script is going to automate the following actions:

  • pull the NSX tags from the source NSX (in VMC)
    • get an authorization token from VMC
    • pull the NSX tags from VMC
  • trigger the HCX migration
    • connect to the local HCX manager (on-prem)
    • get local hcx endpoint information
    • get remote hcx endpoint information
    • get vm information
    • get network information
    • start the migration
  • push the NSX tags to the destination NSX (on-prem)
    • get the nsx vm id
    • push the tags

Let’s start by defining some variables that our script will need:

 1#! /usr/bin/python
 2import requests
 3import json
 4 
 5# Disable Cert Warnings
 6import warnings
 7warnings.filterwarnings('ignore', message='Unverified HTTPS request')
 8 
 9# Source Variables
10nsx_vmc_url = 'https://nsx-****.rp.vmwarevmc.com/vmc/reverse-proxy/api/orgs/****/sddcs/****/sks-nsxt-manager'
11refresh_token = '****'
12vm_to_migrate = 'Tiny Linux VM'
13 
14# Destination Variables
15nsx_dc_url = 'https://nsx.dc.vcn.lab'
16nsx_dc_username = '****'
17nsx_dc_password = '****'
18 
19# HCX Variables
20hcx_url = 'https://hcx-mgr.branch.vcn.lab'
21hcx_username = '****'
22hcx_password = '****'
23dest_network = '****-nsx-ls'

Now, we need to define several functions to implement all the tasks.

First, get an authorization token from VMC:

1## Get the VMC access token
2def GetVmcToken():
3    resp = requests.post('https://console.cloud.vmware.com/csp/gateway/am/api/auth/api-tokens/authorize?refresh_token={}'.format(refresh_token))
4    resp_parsed = json.loads(resp.text)
5    access_token = resp_parsed['access_token']
6 
7    return access_token

Then, get the NSX tags from VMC:

 1# On VMC, Get Tags for the specified VM
 2def GetVmcVmTags(vm_name):
 3    headers = {'Authorization': 'Bearer {}'.format(GetVmcToken())}
 4    # OLD API resp = requests.get('{}/api/v1/fabric/virtual-machines?display_name={}'.format(nsx_vmc_url, vm_name), headers=headers)
 5    # POLICY API
 6    #resp = requests.get('{}/policy/api/v1/infra/realized-state/virtual-machines?enforcement_point_path=/infra/deployment-zones/default/enforcement-points/vmc-enforcementpoint'.format(nsx_vmc_url), headers=headers)
 7    resp = requests.get('{}/api/v1/fabric/virtual-machines?display_name={}'.format(nsx_vmc_url, vm_name), headers=headers)
 8 
 9    vmc_vms = json.loads(resp.text)
10 
11    for vmc_vm in vmc_vms['results']:
12      if vmc_vm.__contains__('tags') and vmc_vm['display_name'] == vm_name:
13        tags = vmc_vm['tags']
14        print('List of VM tags on VMC: {}'.format(tags))
15        return tags   
16 
17    return None

Now that we have the functions to authenticate to VMC and pull the NSX tags, let’s connect to HCX manager to implement the migration:

First, connect to HCX Manager:

 1def ConnectHcx():
 2    print('Connect to HCX')
 3    headers = {'Content-Type': 'application/json'}
 4    payload = {"username": hcx_username, "password": hcx_password}
 5    resp = requests.post('{}/hybridity/api/sessions'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 6 
 7    token = resp.headers['x-hm-authorization']
 8     
 9 
10    return token

Then, get the cloud configuration:

1def GetCloudConfig(token):
2    print('Get Cloud Config')
3    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
4    resp = requests.get('{}/hybridity/api/cloudConfigs'.format(hcx_url), headers=headers, verify=False)
5 
6    resp_parsed = json.loads(resp.text)
7    cloud_config = resp_parsed['data']['items'][1]
8 
9    return cloud_config

Then, get the local endpoint configuration:

 1def GetLocalHcxEndpoint(token):
 2    print('Get Local Endpoint')
 3    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 4    payload = {'filter': {'cloud': {'remote': False}}}
 5    resp = requests.post('{}/hybridity/api/service/inventory/resourcecontainer/list'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 6 
 7    resp_parsed = json.loads(resp.text)
 8    hcx_endpoint = resp_parsed['data']['items'][0]
 9 
10    source = {'computeResourceId': hcx_endpoint['vcuuid'],
11              'endpointId': hcx_endpoint['endpoint']['endpointId'],
12              'endpointName': hcx_endpoint['endpoint']['name'],
13              'endpointType': hcx_endpoint['endpoint']['cloudType'],
14              'resourceId': hcx_endpoint['resourceId'],
15              'resourceName': hcx_endpoint['name'],
16              'resourceType': hcx_endpoint['resourceType'],
17              }
18 
19    return source

And the remote endpoint configuration:

 1def GetRemoteHcxEndpoint(token):
 2    print('Get Remote Endpoint')
 3    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 4    payload = {'filter': {'cloud': {'remote': True}}}
 5    resp = requests.post('{}/hybridity/api/service/inventory/resourcecontainer/list'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 6 
 7    resp_parsed = json.loads(resp.text)
 8    hcx_endpoint = resp_parsed['data']['items'][1]
 9 
10    destination = {'computeResourceId': hcx_endpoint['vcuuid'],
11              'endpointId': hcx_endpoint['endpoint']['endpointId'],
12              'endpointName': hcx_endpoint['endpoint']['name'],
13              'endpointType': hcx_endpoint['endpoint']['cloudType'],
14              'resourceId': hcx_endpoint['resourceId'],
15              'resourceName': hcx_endpoint['name'],
16              'resourceType': hcx_endpoint['resourceType'],
17              }
18 
19    return destination

Now, we need to get the information about the VM to migrate:

 1def GetHcxVm(token, vm_name):
 2    print('Get Hcx Vm')
 3    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 4    payload = {'filter': {'cloud': {'remote': True}}}
 5    resp = requests.post('{}/hybridity/api/service/inventory/virtualmachines'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 6 
 7    resp_parsed = json.loads(resp.text)
 8    hcx_vms = resp_parsed['data']['items']
 9 
10    for hcx_vm in hcx_vms:
11        if hcx_vm['name'] == vm_name:
12            #print(hcx_vm)
13            entityDetails = {'entityId': hcx_vm['id'], 'entityName': hcx_vm['name']}
14            return entityDetails
15 
16    print('\033[91mFail to find the VM...\033[0m')
17    exit(1)
18    return None

and the network used by the VM to migrate:

 1def GetHcxNetworks(token, vm_name):
 2    print('Get Hcx Network')
 3    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 4    payload = {'filter': {'cloud': {'remote': True}}}
 5    resp = requests.post('{}/hybridity/api/service/inventory/virtualmachines'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 6 
 7    resp_parsed = json.loads(resp.text)
 8    hcx_vms = resp_parsed['data']['items']
 9 
10    for hcx_vm in hcx_vms:
11        if hcx_vm['name'] == vm_name:
12            targetNetworks = [{
13                            'destNetworkDisplayName': '{}'.format(dest_network),
14                            'destNetworkHref': '/infra/segments/{}'.format(dest_network),
15                            'destNetworkName': '{}'.format(dest_network),
16                            'destNetworkType': "NsxtSegment",
17                            'destNetworkValue': '/infra/segments/{}'.format(dest_network),
18                            'srcNetworkDisplayName': hcx_vm['network'][0]['displayName'],
19                            'srcNetworkHref': hcx_vm['network'][0]['value'],
20                            'srcNetworkName': hcx_vm['network'][0]['name'],
21                            'srcNetworkType': hcx_vm['network'][0]['type'],
22                            'srcNetworkValue': hcx_vm['network'][0]['value']
23                            }]
24 
25             
26            return targetNetworks
27 
28    print('\033[91mFail to find the network...\033[0m')
29    exit(1)
30    return None

We need to define the VM placement for the migration (basically where the VM is going to be migrated to)

 1def GetHcxPlacement():
 2    hcx_placement = [{
 3                'containerId': "host-16",
 4                'containerName': "esx-03.branch.vcn.lab",
 5                'containerType': "host"
 6                },
 7                {
 8                'containerId': "datacenter-2",
 9                'containerName': "Branch",
10                'containerType': "dataCenter"
11                }]
12    return hcx_placement

And the storage to use at the destination:

1def GetHcxStorage():
2    hcx_storage = {
3                'datastoreId': "datastore-61",
4                'datastoreName': "Datastore",
5                'diskProvisionType': "sameAsSource",
6                'storageProfile': {
7                'type': "default"
8                }}
9    return hcx_storage

Let’s get the VM id from NSX (at the destination)

1# Get the new VM Id
2def GetDcVmId():
3    headers = {'Content-Type': 'application/json'}
4    resp = requests.get('{}/api/v1/fabric/virtual-machines?display_name={}'.format(nsx_dc_url, vm_to_migrate), auth=(nsx_dc_username, nsx_dc_password), headers=headers, verify=False)
5    resp_parsed = json.loads(resp.text)
6    external_id = resp_parsed['results'][0]['external_id']
7    print('DC VM Id:'+external_id)
8 
9    return external_id

And use this Id to push the tags:

1# Tag a VM in the DC
2def TagDcVm(tags):
3    headers = {'Content-Type': 'application/json'}
4    payload = {'external_id': GetDcVmId(), 'tags': tags}
5     
6    print('Tag VM in NSX DC')
7    requests.post('{}/api/v1/fabric/virtual-machines?action=update_tags'.format(nsx_dc_url), data=json.dumps(payload), auth=(nsx_dc_username, nsx_dc_password), headers=headers, verify=False)
8 
9    return None

Finally, a function to start the migration is needed:

 1def HcxMigrate(token, migrations):
 2    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 3    payload = migrations
 4    resp = requests.post('{}/hybridity/api/migrations?action=start'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 5 
 6    resp_parsed = json.loads(resp.text)
 7 
 8    id = resp_parsed['migrations'][0]['migrationId']
 9 
10    print('Start migration...')
11    return id

And another function to follow the progression of the migration:

 1def GetHcxMigrationStatus(token, migration):
 2    headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'x-hm-authorization': token}
 3    payload = {'filter': {'migrationId': [migration]}}
 4    resp = requests.post('{}/hybridity/api/migrations?action=query'.format(hcx_url), data=json.dumps(payload), headers=headers, verify=False)
 5 
 6    resp_parsed = json.loads(resp.text)
 7    state = resp_parsed['items'][0]['migrationInfo']['progressDetails']['state']
 8 
 9    print('Migration State: '+state)
10 
11    return state

Now that’s we’ve defined all our functions, let’s implement the full logic to implement the migration:

 1def main():
 2    print('\033[92mStart migration process...\033[0m')
 3    # Get tags from the VMC vm
 4    tags = GetVmcVmTags(vm_to_migrate)
 5 
 6    # Migrate the VM
 7    token = ConnectHcx()
 8    #print(token)
 9    GetCloudConfig(token)
10    hcx_local_endpoint = GetLocalHcxEndpoint(token)
11    hcx_remote_endpoint = GetRemoteHcxEndpoint(token)
12    hcx_vms = GetHcxVm(token, vm_to_migrate)
13    hcx_networks = GetHcxNetworks(token, vm_to_migrate)
14 
15    hcx_placement = GetHcxPlacement()
16    hcx_storage = GetHcxStorage()
17 
18    input = {
19                'decisionRules': {
20                    'forcePowerOffVm': False,
21                    'removeISOs': True,
22                    'removeSnapshots': True,
23                    'upgradeHardware': False,
24                    'upgradeVMTools': False
25                },
26                'destination': hcx_local_endpoint,
27                'entityDetails': hcx_vms,
28                'migrationType': 'vMotion', # or VR
29                'networks': {
30                    'retainMac': False,
31                    'targetNetworks': hcx_networks
32                },
33                'placement': hcx_placement,
34                'source': hcx_remote_endpoint,
35                'schedule': {},
36                'storage': hcx_storage
37            }
38 
39    migrations = {'migrations': [{'input': input}]}
40    migration_id = HcxMigrate(token, migrations)
41 
42    while True:
43        status = GetHcxMigrationStatus(token, migration_id)
44        if (status == 'COMPLETED'):
45            print('\033[92mMigration completed...\033[0m')
46            break
47        if (status == 'FAILED'):
48            print('\033[91mMigration failed...\033[0m')
49            break
50        time.sleep(15)
51 
52    # Retag the VM in the DC
53    TagDcVm(tags)
54 
55    print('\033[95mMigration process done.\033[0m')
56     
57  
58 
59main()

Now, let’s test our script. Here is the VM at source, before the migration: VM at source, before the migration

Some tags are applied to the VM tags at source, before the migration (scope = scope1, tag = tag1)

Start our script:

 1adeleporte@adeleporte-a01 hcx-tags % ./main.py
 2Start migration process...
 3List of VM tags on VMC: [{u'scope': u'scope1', u'tag': u'tag1'}]
 4Connect to HCX
 5Get Cloud Config
 6Get Local Endpoint
 7Get Remote Endpoint
 8Get Hcx Vm
 9Get Hcx Network
10Start migration...
11Migration State: PROCESS_COLLECT_GATEWAY_DETAILS
12Migration State: PROCESS_COLLECT_GATEWAY_DETAILS
13Migration State: PROCESS_COLLECT_GATEWAY_DETAILS
14Migration State: PROCESS_COLLECT_VM_DETAILS_RESULT
15Migration State: PROCESS_COLLECT_VM_DETAILS_RESULT
16Migration State: JOIN_RECONFIGURE
17Migration State: PROCESS_RECONFIGURE_RESULT
18Migration State: DESTINATION_SIDE_START_RELOCATE
19Migration State: PROCESS_START_RELOCATE_RESULT
20Migration State: PROCESS_DESTINATION_SIDE_RELOCATE_PROGRESS_RESULT
21Migration State: PROCESS_SOURCE_SIDE_RELOCATE_PROGRESS_RESULT
22Migration State: SOURCE_SIDE_CLEANUP
23Migration State: DESTINATION_SIDE_CLEANUP
24Migration State: COMPLETED
25Migration completed...
26DC VM Id:500c9196-a9db-34c9-237d-afb214161169
27Tag VM in NSX DC
28Migration process done.

The migration is started and the HCX console can be used to follow its progress.

HCX Migration

When the migration is finished, let’s connect on the DC NSX to check the tags

HCX Migration Tags after migration

DISCLAIMER: The views and opinions expressed on this blog are our own and may not reflect the views and opinions of our employer