Over the last few months I’ve done a couple of VMware Cloud Foundation (VCF) 3.x deployments and the question I almost always get is how to perform a startup or shutdown of the VCF environment. Since the documentation for VCF 3.0 and 3.5 doesn’t give you any guidance on how to do this, I set out to create one so I don’t have to do it each time doing a deployment. This post covers the startup/shutdown procedure for VCF 3.x.
Since the VCF architecture is based upon the VMware Validated Design (VVD) architecture, the startup/shutdown procedure for the VVD is a good starting point. But there are some differences in the two architectures. For example, the VVD uses a load-balanced Platform Services Controller configuration where as VCF does not.
Also, the documentation for VCF 2.x does provide a startup/shutdown procedure. Since the biggest change in VCF 3.x compared to VCF 2.x is the introduction of the Bring-Your-Own-Network construct, the startup/shutdown procedure for the virtual infrastructure hasn’t changed that much.
In the end I combined both the VVD 4.3 and VCF 2.x documentation to provide a startup/shutdown procedures for VCF 3.x.
Note; VCF 3.0 is based on VVD 4.3 and VCF 3.5 is based on VVD 5.0. Since the overall architecture hasn’t changed, I used the VVD 4.3 documentation.
Note; The startup/shutdown procedures are almost the same when using VCF 3.5 with a NSX-v or NSX-T workload domain. The procedures below describe the steps for NSX-v and NSX-T workload domains.
Startup a VMware Cloud Foundation Environment
Start up the virtual machines of the management stack by following a strict order to guarantee the faultless operation of and the integration between the components.
Prerequisites
- Verify that you have direct console access to the ESXi hosts in the system.
- Verify that you have the host names and IP addresses of the ESXi hosts that are members of the management domain.
- Verify that you have the host names and IP addresses of the ESXi hosts that are members of each workload domain.
- Verify that external dependencies for the environment, such as, Active Directory, DNS, and NTP are available.
- Refer to VMware Knowledge Base article 2142676 for information on starting up hosts and exiting maintenance mode.
Procedure
- Power on each ESXi host in the management domain, and exit maintenance mode.
- Power on each ESXi host in the first workload domain, and exit maintenance mode.
- Repeat this step for each workload domain.
- Power on the infrastructure management VMs in the management domain.
- Wait until each VM is powered on and all it services started before powering on the next VM.
- Power on the VMs in the following order:
- Platform Services Controller virtual appliances
- vCenter Server for the management domain
- vCenter Server for each workload domain.
- SDDC Manager
- NSX Manager virtual appliance for the management domain
- NSX Manager virtual appliance for each workload domain
- NSX Controller cluster virtual appliances for the management domain
- Optional: NSX Load Balancer virtual appliances for the management domain (only when vRealize Operations and/or vRealize Automation is deployed)
- Power on the vRealize Automation virtual appliance and IaaS components in the management domain.
- Wait until each VM is powered on and all it services started before powering on the next VM.
- Power on the VMs in the following order:
- The vRealize Automation IaaS SQL Server VM(s)
- All vRealize Automation virtual appliances
- All vRealize Automation IaaS Web Server VMs
- (Power on the primary IaaS Web Server VM first)
- All vRealize Automation IaaS Manager Services
- (Power on the primary IaaS Manager Server VM first)
- All vRealize Automation IaaS proxy agents
- All vRealize Automation IaaS Distributed Execution Management (DEM) hosts
- Power on the vRealize Operations Manager virtual appliances in the management domain.
- Wait until each VM is powered on and all it services started before powering on the next VM.
- Power on the VMs in the following order:
- The vRealize Operations Manager master node
- The vRealize Operations Manager master replica node
- All vRealize Operations Manager data nodes
- Power on the vRealize Log Insight virtual appliances in the management domain.
- Wait until each VM is powered on and all it services started before powering on the next VM.
- Power on the VMs in the following order:
- The vRealize Log Insight master node
- All vRealize Log Insight worker nodes
- Power on the vRealize Suite Lifecycle Manager virtual appliance in the management domain.
- Power on the VMs in the first VI workload domain..
- Each workload domain includes a three-node NSX Controller cluster. Power on these VMs first.
- Power on the NSX Edge Services Gateways and Logical Router VMs (NSX-v).
- Repeat this procedure on each workload domain.
- Power on the NSX Edge Cluster Nodes (NSX-T)
- Power on the workload domain VMs.
- Repeat this procedure on each workload domain.
- Each workload domain includes a three-node NSX Controller cluster. Power on these VMs first.
Shutdown a VMware Cloud Foundation Environment
Shut down the virtual machines of the management stack by following a strict order to avoid data loss and faults in the components.
Prerequisites
- Coordinate the shutdown in advance with business stakeholders to minimize any impact.
- Verify that you have saved the account passwords to a location external from the Cloud Foundation system you are shutting down.
- Verify that valid backups of all management and tenant VMs are available and saved to a location external from the Cloud Foundation system you are shutting down.
- Verify that you have direct console access to the ESXi hosts in the system.
- Verify that you have the host names and IP addresses of the ESXi hosts that are members of the management domain.
- Verify that you have the host names and IP addresses of the ESXi hosts that are members of each workload domain.
- Refer to VMware Knowledge Base article 2142676 for information on shutting down hosts and entering maintenance mode.
Procedure
- Shut down the workload VMs in each workload domain.
- Power off the workload domains VMs.
- Repeat this procedure on each workload domain.
- Power off the NSX Edge Services Gateways and Logical Router VMs (NSX-v)or the NSX Edge Cluster Nodes (NSX-T).
- Repeat this procedure on each workload domain.
- Each workload domain includes a three-node NSX Controller cluster. Shut down these VMs last.
- Repeat this procedure on each workload domain (NSX-v).
- Power off the workload domains VMs.
- Shut down the vRealize Suite Lifecycle Manager virtual appliance in the management domain.
- Shut down the vRealize Log Insight virtual appliances in the management domain.
- Wait until each VM is powered off and all it services stopped before powering off the next VM.
- Shut down the VMs in the following order:
- All vRealize Log Insight Worker nodes
- The vRealize Log Insight Master node
- Shut down the vRealize Operations virtual appliances in the management domain.
- Wait until each VM is powered off and all it services stopped before powering off the next VM.
- Shut down the VMs in the following order:
- All vRealize Operations Manager data nodes
- The vRealize Operations Manager master replica node
- The vRealize Operations Manager master node
- Shut down the vRealize Automation virtual appliances in the management domain.
- Wait until each VM is powered off and all it services stopped before powering off the next VM.
- Shut down the VMs in the following order:
- All vRealize Automation IaaS Distributed Execution Management (DEM) VMs
- All vRealize Automation IaaS Proxy Agent VMs
- All vRealize Automation IaaS Manager Server VMs
- Shut down the secondary IaaS Manager Server VM first; shut down the primary IaaS Manager Server VM second
- All vRealize Automation IaaS Web Server VMs
- Shut down the secondary IaaS Web Server VM first; shut down the primary IaaS Web Server VM second
- All vRealize Automation virtual appliances
- The vRealize Automation IaaS SQL Server VM
- Shut down the infrastructure management VMs in the management domain.
- Wait until each VM is powered off and all it services stopped before powering off the next VM.
- Shut down the VMs in the following order:
- Optional: NSX Load Balancer virtual appliances for the management domain (only when vRealize Operations and/or vRealize Automation is deployed)
- All NSX Controller cluster virtual appliances for the management domain.
- The NSX Manager virtual appliances for the workload domains
- The NSX Manager virtual appliance for the management domain
- SDDC Manager VM
- The vCenter Server virtual appliance for the workload domains
- The vCenter Server virtual appliance for the management domain
- The Platform Services Controller virtual appliances
- Place the hosts for each workload domain in maintenance mode, and power off.
-
- Repeat this procedure on each workload domain.
-
- Place the hosts for the management domain in maintenance mode, and power off.
- Shut down the unassigned ESXi hosts in the Cloud Foundation system, if any.
Startup/Shutdown Order of the Management VMs
The following table provides an overview of the startup and shutdown order of all the components in the environment.
| Component | Startup Order | Shutdown Order |
| ESXi Hosts | 1 | 7 |
| ESXi Hosts Management Domain | 1 | 3 |
| ESXi Hosts Workload Domain # | 2 | 2 |
| Infrastructure Management | 2 | 6 |
| Platform Services Controller #1 | 1 | 6 |
| Platform Services Controller #2 | 1 | 6 |
| vCenter Server Management Domain | 2 | 5 |
| vCenter Server Workload Domain # | 2 | 5 |
| SDDC Manager | 3 | 4 |
| NSX Manager Management Domain | 4 | 3 |
| NSX Manager Workload Domain # | 4 | 3 |
| NSX Controller Cluster Management Domain (#1, #2, #3) | 5 | 2 |
| Optional: NSX Load Balancer Management Domain (#0, #1) | 6 | 1 |
| vRealize Automation | 3 | 5 |
| vRealize Automation IaaS SQL Server VM(s) | 1 | 7 |
| vRealize Automation Appliances | 2 | 7 |
| vRealize Automation IaaS Web Servers #1 | 3 | 6 |
| vRealize Automation IaaS Web Servers #2 | 4 | 5 |
| vRealize Automation IaaS Manager Servers (#1) | 5 | 4 |
| vRealize Automation IaaS Manager Servers (#1) | 6 | 3 |
| vRealize Automation IaaS Proxy Agents (#1, #2) | 7 | 2 |
| vRealize Automation IaaS DEM Hosts (#1, #2) | 7 | 1 |
| vRealize Operations | 4 | 4 |
| vRealize Operations Manager Master Node | 1 | 3 |
| vRealize Operations Manager Replica Node | 2 | 2 |
| vRealize Operations Manager Data Node(s) | 3 | 1 |
| vRealize Log Insight | 5 | 3 |
| vRealize Log Insight Master Node | 1 | 2 |
| vRealize Log Insight Worker Nodes (#1, #2) | 2 | 1 |
| vRealize Suite Lifecycle Manager | 6 | 2 |
| vRealize Suite Lifecycle Manager | 1 | 1 |
| Workload Domain | 7 | 1 |
| NSX Controller Cluster Workload Domain # (#1, #2, #3) | 1 | 3 |
| NSX Edge Service Gateways and Logical Router VMs (NSX-v) | 2 | 2 |
| NSX Edge Nodes (NSX-T) | 2 | 2 |
| Workload Domain # VMs | 3 | 1 |
verbrough 
Leave a Reply