vROPS RBAC and upgrades

Last week I was at a customer doing some vROPS magic, which included updating the current vROPS 6.4 cluster to 6.5 to get the improved vRLI integration. Upgrading both vRLI and vROPS clusters went perfect, but the vRLI integration items like the Log Insight icon, and Logs tab would not be visible in vROPS after the configuration.

We followed the configuration steps as described in the documentation and verified all the configuration steps. After verifying all the configuration steps and coming to the conclusion that the configuration is correct, we tried logging in with the default admin credentials instead of the customer Active Directory credentials. With the admin account, the Log Insight icon and Logs tab were visible!

When upgrading vROPS to a version with new features, these new features get new privileges to be assigned. When using the default roles, these privileges get assigned automatically to the applicable roles, like for instance the administrator role. But if you create a new role, by cloning are just creating a new, these privileges need to be assigned manually.

After assigning the new privileges to the created role, the Log Insight page and Logs tab were finally visible for the customer account.

TLDR; When upgrading vROPS to a higher version with new features, make sure to check the privileges for these new features on roles. User created roles do not automatically receive newly added privileges.

vROPS 6.5 Node Specifications

VMware has released version 6.5 of vROPS which includes a new node configuration type for additional monitoring capabilities. From the release notes:

Additional monitoring capabilities

  • Adds ability to increase memory and increase the scope of monitoring within the same environment.
  • Enables you to monitor larger environments with the same footprint through platform optimization.
  • XL size node enables you to monitor more objects and it processes more metrics.

The previous largest configuration for an analytics cluster node was a 16 vCPU, 48 GB Memory node. This node configuration was for environments larger then 4.000 VM’s.

Apparently this was not enough 😉

The new largest node (XL) configuration is a little bit bigger.. 24 vCPU and 128 GB Memory! This node configuration is for environments between 12.000 and 40.000 VM’s.

The VMware documentation has not been updated with this configuration yet. I’ve downloaded the OVA, extracted it, and opened the OVF to get these specifications.

vrops_xl_config

The new specifications for all the node configurations are:

Node Size vCPU Memory
Extra small 2 8 GB
Small 4 16 GB
Medium 8 32 GB
Large 16 48 GB
Extra Large 24 128 GB

–Update–

thanks to @pheldoorn for commenting the sizing guidelines. The maximum cluster size with Extra Large sized nodes, is 4 nodes and not 16 nodes as with Medium and Large sized nodes.

 

vROPS 6.4 – Metric Config Picker

With vRealize Operations Manager you can create a metric configuration file. This is an XML file that contains predefined metrics that can be used in different widgets. With the XML file you can skip the process of manually picking the required metrics and attributes over and over again. Another advantage is that you can reuse this XML file in different widgets for different objects in your environment.

The metric configuration file contains 3 important tags:

<Adapterkind>

The AdapterKind tag determines which adapter is required for the metrics you want to include.

<ResourceKind>

The ResourceKind tag determines the resource type, e.g. cluster, datastore, virtual machines, host.

<Metric attkey>

The Metric attrkey tag determines which attributes and/or metric are included in the configuration.

In previous versions of vROPS if you wanted to create a metric configuration file you needed to know the true adapter kind name , resource kind name  and metric value name in vROPS instead of the display names. You could get these names using the API browser of vROPS. An excellent blog post on how to get these names can be found here.

In vROPS 6.4 they made life a bit easier, it is now possible to use a metric picker to get the right metric in you metric configuration file. Depending in which line you are in the metric configuration file, the applicable metric picker will be available.

First create the metric configuration file and select the applicable adapter kind. In my case I only have the vCenter Server solution configured.

vrops_adapterkind_picker_menu

Next select the resource kind of which you want to use the metrics.

vrops_resourcekind_picker_menu

And finally, select the metrics you want to add to you metric configuration file.

vrops_metric_picker_menu

The metric configuration file is now configured and can be used in widgets. More information on how to use the metric configuration file in a widget can be found here.

I personally think this is a very welcome addition to the vROPS GUI. Creating a metric configuration file used to take a lot of time looking up the name of the metric you wanted to use. This will save you time and maybe even some frustration when you had found out that the metric you selected was not the right one.

Update: Forgot to mention my coworker Johan van Amersfoort for discovering this new feature. Visit his blog on vhojan.nl for EUC awesomeness!

 

 

vROPS 6.4 – New Dashboards

VMware has released vROPS 6.4 which contains several new dashboards to display status and identify problems. The new dashboards can be divided into several categories:

  • Environment and capacity overview dashboards to get a summary of your environments.
  • VM troubleshooting dashboard that helps you diagnose problems in a VM and start solving them.
  • Infrastructure capacity and performance dashboards to view status and see problems across your datacenter.
  • VM and infrastructure configuration dashboards to highlight inconsistencies and violations of VMware best practices in your environment

In this post I will highlight some of these dashboards. If you want to know more about all the other dashboards I suggest you download and install or upgrade to vROPS 6.4 🙂

Operations Overview

This dashboard provides an general overview of your vSphere environment such as amount of VM’s, clusters hosts, and datastores. The dashboard also provides top list information about virtual machines with CPU contention, memory contention or disk latency.

vROPS Dashboard - Operations Overview

Capacity Overview

This dashboard provides an overview of the capacity of your vSphere environment such as total CPU cores, memory and storage capacity. The dashboard also provides graphs for the different resources utilization containing realtime and trend/forecast data.

vROPS Dashboard - Capacity Overview

Troubleshoot a VM

This dashboard provides general troubleshooting information for a virtual machine such as critical alerts and possible contention.

Maybe the most requested dashboard by customers and I am excited this is now default available in vROPS!

vROPS Dashboard - Troubleshoot a VM

Heavy Hitter VMs

Like the name suggests, this dashboard provides information of the top heavy virtual machines in your vSphere environment such as top highest IOPS and network throughput.

vROPS Dashboard - Heavy Hitter VMs

Cluster Performance

This dashboard provides general performance information for clusters such as critical alerts and possible contention.

vROPS Dashboard - Cluster Performance

ESXi Configuration

This dashboard provides general information about the hardware of the vSphere hosts in your environment such as hardware model, ESXi version and power management setting.

Not in the picture below but the dashboard also provides an overview of the configuration of all the vSphere hosts. This overview contains information such as CPU sockets, NICs, Power State, CPU Model, etc.

vROPS Dashboard - ESXi Configuration

VM Usage

This dashboard provides general information about virtual machines in your environment such as general virtual machine configuration and graphs about CPU, memory and IOPS demand.

vROPS Dashboard - VM Usage

The addition of these dashboards are very welcome and I think these dashboards make vROPS even better to use. Here are some links to vROPS resources.

Release notes

http://pubs.vmware.com/Release_Notes/en/vrops/64/vrops-64-release-notes.html

vROPS Sizing Guidelines

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2147780

Download

https://my.vmware.com/en/web/vmware/info/slug/infrastructure_operations_management/vmware_vrealize_operations/6_4

 

vROPS 6.3 – vSphere Hardening Guide 6.0

As described in my previous post I have upgraded my lab vROPS cluster to vROPS 6.3. After a couple of days I finally had time to look at the updated vROPS policies. One of the things I was most interested in was support for the vSphere 6.0 hardening guide.

With vROPS 6.3 it is possible to generate alerts when a host or vCenter violates rules found in the vSphere 6.0 hardening guide. In previous releases only the vSphere 5.5 hardening guide could be used.

To enable alerts for the vSphere hardening guide you need to perform the following actions:

  • Enable vSphere hardening guide alerts in the VMware vSphere solution
  • Customize a policy to enable the vSphere hardening guide alerts

To enable vSphere hardening guide alerts in the VMware vSphere solution define the monitoring goals: Administration -> Solutions -> VMware vSphere -> Configure -> Define Monitoring Goals.

vrops_enable_hardening_alerts

After this you need to customize your policy to enable the alerts. At this step I had a problem with the vSphere hardening guide alerts. Because I performed an upgrade and I did not want to lose any customization on default objects I choose to not reset out of the box content during the upgrade.

This resulted in the policies not being updated with the new vSphere hardening guide alerts.

vrops_hardening_guide_55

After some digging I found a VMware KB article explaining that the policies were not updated because of my choice to not reset out of the box content. The only solution is to reset default content in the VMware vSphere Solution. You can do this via Administration -> Solutions -> VMware vSphere -> Reset Default Content.

Keep in mind that this removes all your customizations on default objects such alert definitions, symptoms, policy definitions and dashboards.

vrops_reset_default_content

A common best practice is to not customize the out of the box content but clone or create new objects such as dashboards and policies.

After resetting the default content I could enable the vSphere 6.0 hardening guide alerts in the policy I have created and alerts where created for the hosts.

vrops_hardening_guide_60

vrops_alert_hardening_guide

 

 

 

vROPS 6.x Blank Dashboard

A few weeks ago I upgraded the vROPS cluster in the lab environment to vROPS 6.3. The cluster is a 3 node cluster with a master, replica and remote collector. The cluster is behind a NSX load balancer to provide a single FQDN to connect to the cluster.

The upgrade went smoothly and all nodes were upgraded without a problem. One important thing to remember is to always update the virtual appliance OS first before upgrading vROPS! If you do not do this, you will break your vROPS instance!

But the problem started when I logged on to the vROPS instance. Some dashboard were working fine but other dashboards would not show any contect. The content on these faulty dashboards varied from completely blank to only some widgets.

vROPS Blank Dashboard

At first I thought the problems appeared only on custom dashboards I had made but after looking at some more dashboards it appeared that it also happened on the out of box dashboards.

At this stage I was thinking I broke the vROPS cluster and needed to redeploy the cluster. But before I was sure I really needed to do this I asked a colleague if he experienced the same problems. He could view the dashboards without any problem.

Because of this I tried to open the dashboards from another browser, in this case Internet Explorer instead of Chrome. The dashboards were working fine with Internet Explorer, so my first suspicion went to Chrome. But my colleague had opened the dashboards in Chrome and they worked fine for him.

Eventually my colleague suggested that I cleared the Chrome browser cache so I would not have any old references to the dashboards. And lo and behold, the dashboards were working fine after this!

vROPS Dashboard

TLDR, if you have any issues with content on vROPS dashboards clear your browser cache first 🙂