Migrate for Compute Engine | AWS →GCP

In this article, I will demonstrate “How to migrate AWS EC2 Instance to Google Cloud’s Compute Engine VM instance using Google’s native Migrate for Compute Engine Migration tool”.

VM Migration from AWS to GCP using Migrate for Compute Engine

In this rapidly growing technological world, Most companies are refactoring their infra to leverage the benefits of Machine learning, AI & Bigdata solutions, and Google Cloud Platform is the Leading Cloud Provider for modern technological solutions. Migrating workload virtual Machines is the first step to getting started with GCP & this may be a little bit of a blurring process for newcomers. Migrate for Compute Engine is one the most popular & Google’s native migration tools to achieve this & in this article, we’ll deep dig out into this tool while migrating demo EC2 instance to GCP.

Steps

  1. VPN Setup between AWS & GCP Environment
  2. Configuring required firewalls in GCP
  3. Creating Migration Manager in Migrate for Compute Engine
  4. Setting up AWS Environment
  5. Migration Agent Installation in AWS VMs
  6. Migration

Let’s start the learning……..

Step 1: Creating a VPN between the VPC’s of AWS and GCP

For setting up your VPN, follow the below doc, in which you’ll find step-by-step instructions.

Link: VPN setup between AWS & GCP by Voval Jain

Step 2: Setup Firewall Configurations needed on the GCP side

Please follow the link for all firewall rules with detailed information.

Step 3: Creation & configuring migrate for compute engine Migration Manager

Navigation menu → compute engine → migrate on compute engine → try the new version

Fill the all required fields according to your specifications.

Note:- Here we’re going to create two service accounts, so be sure that you’ve required roles with you so that you can create these service accounts.

  1. Migration Service account
  2. Migration Cloud Extension Service account

Don’t worry about what permissions are required by these service accounts. GCP console will automatically add the permissions to the service account if you’re creating new ones, so just click create a new survive account.

After adding a service account, your screen will look like this. Also, you can see what the permissions are going to use in the migration process.

At last, you’ve to create one user named apiuser you create that one also. Keep this password with you, you’ll need ID & Password for logging into the migration console & in the process of migration agent installation in AWS VM’s which are going to migrate to GCP (Especially Windows VM)

You can create a service account alone as well from IAM Console or CLI, The scripts mentioned in the given link provides the command to create the service account which will be required for the VM migration process.

Note:- For the workloads (to be migrated) to use in a later step, create a separate service account other than mentioned in the link.

Now just review & create the migration manager

You can see the same after the manager created, note the internal IP of the Migration Manager, we will use this IP address when installing the migration agent in the AWS Windows VMs.

Step 4: Setup AWS environment for Migrate for Compute Engine (Velostrata):

Required Policies: IAM – CreateGroup, PutGroupPolicy, CreateUser, AddUserToGroup

  1. Create an IAM group named VelosMgrGroup for use by a Migrate for Compute Engine service account. This group enforces an access policy with the minimum privileges required by Migrate for Compute Engine and allows provisioning and monitoring of cloud-side components and worker VMs.
  1. To set the AWS credentials, we have to create a new stack in the Cloud Formation, providing the JSON file available on the link below. After the stack is created, a velostrata IAM group is generated with all the required permissions.
The following links→ download page → CloudFormation Stack

Note:- Here, We’re using velostrata version 4.11.9 so if you’re using a different version then download the proper stack accordingly from the download page.

After downloading, extracting the file you’ll get the JSON file like this VxCF-GA-V3-IAMONLY.rev1.json

In the AWS Management Console

Search → cloud formation → create stack → with existing resources(import resources)

Click on the upload file & upload your JSON file you extracted in the above step.

Just specify some more details like VPC in further steps & create your stack.

3. Now create an IAM user with programmatic access and add it to the newly created Velostrata Group (VelosMgrGroup).

AWS Console -> Click on Account Name -> Security Credentials -> left menu -> users -> Create New User

Now, add the new user to the Velostrata Group (VelosMgrGroup)

Note: Be sure that you’ve given the Programmatic access to the new user, Also save the Access key & Secret key somewhere, we need both keys in the further process.

Step 5: Installation of the agent in AWS VMs (Which will be migrate)

On the VM that we want to migrate from AWS to GCP, we need to install the prep package according to our OS. In the below example, we have shown different operating systems.

Note:- before starting migration agent/package installation, check the connectivity using private IP

Installation in windows EC2 Instance:

  1. Connect to your Windows instance using RDP
  2. Open the browser & download the file from the below link & install it
  3. Run your Windows PowerShell as an Administrator & commands which are given below

Link: Windows Velostrata Agent Package Download URL

Import-Module Velostrata.PowerShellConnect-VelostrataManager

Important: This will prompt you for login so keep your migration manager VM’s internal IP & remember we have created one user called apiuser & their password, so we’re going to use that details here.

This will download some packages, so be patient.

Note:- This is for velostrata version 4.11.9 only, so according to your version, download the file.

Link: https://cloud.google.com/migrate/compute-engine/docs/4.11/resources/downloads

Migrate for compute engine downloads section → click PowerShell Module

Reference Links:

Installation in Ubuntu EC2 Instance :

Download the migration agent package & install it on the machine.

wget https://storage.googleapis.com/velostrata-release/4.11.9/migrate-for-gce-prep-4.11.9-0.debsudo apt-get updatesudo apt --fix-broken installsudo apt install multipath-tools*sudo dpkg -i migrate-for-gce-prep-4.11.9–0.debsudo apt-get update && sudo apt-get install -f -y

Now, we’re ready for the migration process

Important: To access velostrata console, you’ve to add your PC’s IP to the firewall rule velos-webui.

Search for your IP in the browser → note your IPv4 address & add to velos-webui firewall rule as source IP ranges <Your IP>/32

Step 6: Migration

GCP Console → Navigation menu → migrate for compute engine → click on your migration instance → Instance details → you’ll redirect to compute engine → open your migration manager VM's external IP address in a new tab.

Note:- If you get some error then in the page click on advanced options & proceed to <vm external ip> (unsafe).

Click on Advanced & then Click on Proceed to unsafe

After getting the UI, we get the page to enable logs for the StackDriver, we can enable them.

Migration Procedure:

6.1: Set up the ‘Source Cloud’ section with (AWS) credentials

Here, we need to provide the source credentials from the AWS IAM user credentials generated for velostrata in Step 4 (Setup AWS IAM for Migrate on compute Engine (Velostrata)).

For adding the source cloud credentials,

Here you’ll need to give User ID & Password so user apiuser as username & their password, we’ve created at the time of migration manager VM creation.

Source Cloud → Cloud Credentials → Add the details like region, access and secret key.

After the cloud credentials are created, go to the source cloud then cloud details

Note:- In this section, select the proper subnet that virtual Machines are running on the AWS side

6.2: Set up the ‘Target Cloud’ section:

Click home → Target Cloud

In the Target Cloud, we create the Cloud Extensions.

Now, creating the Cloud Extensions requires filling 4 sections as given below:

6.2.1 Network Setting:

Here,

a. Select your GCP Project, Region, and VPC from the dropdown.

b. Specify the network tags ‘fw-velostrata’ for the edge nodes and ‘fw-workload’ for workloads to be migrated.

c. Select the GCP project from the dropdown

d. In the ‘Service Account for Workload’ select the service account specially meant for a specific VM service or workload.

f. leave other fields as default.

6.2.2 Cloud Extension:

  1. Name of the cloud extension of any choice
  2. Service account created for the edge nodes at the time of migration manager VM creation.
  3. Select size:

6.2.3 Zones

  1. Two cloud extension edge nodes will be created in the process of migration. In order to have HA in place, select two different zones of the region.
  2. Select the same subnet for both zones to reduce complexity.
  3. The workload subnet is the subnet where the VM will be migrated to. Select accordingly as planned for the same.

6.2.4 Labels:

Labels are tags that will be attached to all the VMs created after running the migration wave (workload + edge nodes).

Velostrata edge node created in the GCP environment

Once cloud extension is done, enable HTTP and HTTPS firewall for edge nodes

Common Issue with Cloud Extension:

State: Impaired.

There could be various reasons, but very common is “could not connect to null instance on port 443”

Solution: Check the firewall rules and verify if necessary permission is given to the service account used.

6.3: Setting up Migration waves

Home → migration waves

Migration Waves is a four-step process:

  1. Generate Runbook csv (containing workload info)
  2. Modify the CSV file as per need
  3. Create Wave for Migration using the CSV
  4. Validate and Start Migration.
  5. Generate Runbook CSV

In the above fields:

  1. Source: Source from where VM is to be migrated. (E.g. AWS, Azure)
  2. Source Cloud Details: The credentials which were generated in the source cloud and from that cloud details which have been created.
  3. Source Tag: The tags given to the source AWS/Azure VMs, multiple tags can be added.
  4. Target Cloud Extension: The Cloud Extension created which is to be used is mentioned here.
  5. Target Network: Tick the option (Populate with Cloud Extension Defaults)

After clicking on the “Create” button, a CSV file will be downloaded which contains all details of the AWS VM to be migrated.

The next step will be editing the CSV file as mentioned earlier and creating a ‘wave’

B. Modify CSV file as per need

Few edits are to be performed on the generated CSV and save in your local system.

  1. The values in the first column in the CSV are negative, we have to suppress the negative sign.
  2. We have to manually enter the machine type (GCP) in the ‘TargetInstanceType’ column. E.g. n1-standard-1
  3. Give a name to the instance

Below is the sample runbook file

C. Create Wave for Migration using the CSV

Provide the name and upload the generated runbook CSV file here.

D. Validate and Start Migration.

Once the wave is created using the runbook CSV, it has to be validated and only after validation we can perform the jobs of migration.

Once validated successfully, You’ll get passed status. Now, various types of migration can be performed by clicking on the ‘New Job’

In the operation, Choose a Migration method as I’ve chosen full migration for this demo but you can choose according to your need & requirements.

VM Migration Life Cycle:

1. Run-in-cloud –This operation moves the source VMs from AWS to GCP. This method does not completely move VM storage to the GCP. The run-in-cloud operation performs the following actions:

Key Points to Note:

2. Full Migration – The full migration operation moves VMs in one step from source to target. In doing so, it

Key Points to Note:

Once the full migration is done and shows the ready to detach status then go to the action then detach once detach is done go to action then clean up

Importer server in AWS:

AWS moving to GCP cloud

2. Aws instances are migrating to GCP:

AWS migrated Ubuntu Server login in GCP:

[Failed to prepare VM to detach] (Exporter is terminated)

6. Cleanup – After the VMs are detached, and you have completed any required validation, you can start to detach cleanup. Each VM is then marked as unmanaged by Migrate for Compute Engine.

Other Migration Jobs available with Migrate for Compute Engine

7. Test Clone

Note:- Test clones are available only for migrations from on-premises data centres to GCP. Test clones are not available for cloud-to-cloud migrations.

Conclusion

In this article, we had successfully migrated our EC2 instance ( Windows / Linux Server) to Google Cloud Platform compute engine Instance using Google Cloud’s native migration tool Migrate for compute Engine formally Velostrata. I hope this will give a better understanding of the Migrate for Compute Engine migration tool & the process we follow during the migration.