vCSA, root partition is (almost) full

18/02/2018

hwA short post on a topic that I recently experienced on vCenter Server Appliance, version 6.0.
After receiving an alert that the root “/” partition was quickly filling up, it is time to act quickly. When the root partition reaches 100% of it’s capacity, service disruption can occur.
First step is to check the capacity of the vCSA partitions. Log in to the vCSA through SSH, if you are running the appliance shell, enable and access the Bash shell:

Command> shell.set --enabled true
Command> shell

In the Bash shell run this command to check the capacity of the partitions:

# df -h

The second line of the output (starting with /dev/sda3) shows the status of the root partition. If the value under Use% reaches 100%, you are in trouble. Also notice that the root partition is only 11 GB.
Second step is to determine the root cause of the full partition. A good strategy is to look for large consumers. The next command searches for files larger then 100 MB, only on the root partition:

# find / -xdev -type f -size +100M

In my case some interesting results:

/usr/lib/vmware-sca/wrapper/bin/wrapper.log
/usr/lib/oracle/11.2/client64/lib/libociei.so
/var/log/dnsmasq.log-20180121
/var/log/dnsmasq.log-20180128
/var/log/dnsmasq.log-20180107
/var/log/dnsmasq.log-20180114
/var/log/dnsmasq.log
/etc/vmware-vpx/docRoot/client/Vmware-viclient.exe

The most eye-catching files are: the wrapper.log and the dnsmasq.log files.

Read the rest of this entry »


Getting started with the vCSA 6.x – Part 3

22/01/2018

In part 1 and part 2 of this series about the vCSA, we have covered topics like; the shells, filesystem, services, health, logging, database and some extra tools. Recently I realised there a few more topics worth mentioning.

Appliance MUI

In pre 6.0 releases of the vCSA, there was a vCenter Server Appliance Management Interface, better known as the VAMI. This management interface is written in HTML5 and is now called the e Appliance Management User Interface (Appliance MUI).

You will find the new management interface in vCSA 6.0 and 6.5, however there are some differences.

You can login to this interface, using: https://<vCSA fqdn or IP>:5480. Us a local account such as the “root” account.

Fig. 1 – Summary vCSA 6.0.

Read the rest of this entry »


Getting started with the vCSA 6.x – Part 2

30/07/2017

In the previous post we started to unravel the vCSA and discussed topics like the Appliance shell, the file system and the services. In this post we will continue with the vCSA Health.

Health

Knowing the health of your system is important. Like the Windows vCenter Server, the vCSA is also able to report its health. Most common is using the vSphere Web Client and from the main menu, choose: System Configuration and watch the “Service Health” pane. Detailed information can be found by clicking on the various Services.

Figure 1

However, from the Appliance shell, the following API command will also inform you”

Command> system.health.get

If everything is OK, it will report; Health: green

In the Bash shell, you can browse to the folder: /etc/vmware-sca/health/.
On a vCSA 6.0, you will find two files with health status information:

vmware-vpxd-health-status.xml
vmware-postgres-health-status.xml

On a vCSA 6.5, you will only find the first file.

Read the rest of this entry »


Getting started with the vCSA 6.x – Part 1

12/07/2017

The vCenter Server Appliance is the new vCenter Server. In the old days, we had a brand new Windows Server on which the vCenter Server was installed. The necessary database server was quite often an external MS SQL database and sometimes an internal database. In the those days, tweaking the Windows Server and the installed components was more or less a common practice, due to the familiarity with Windows.

But now the vCenter Server Appliance (vCSA) is the new and preferred standard. The vCSA comes as a virtual appliance and is ready to run within minutes compared to the old vCenter Server. Although a (virtual) appliance still means; (virtual) hardware, an operating system, middleware and applications, VMware likes to treat the vCSA as a “black box”. Like most appliances, the operating system is a Linux flavor and you can log in. After a successful log in, you will encounter the first discouragement; you are not welcomed by a Bash shell but with the default “Appliance Shell”. For more information read my post on the vCSA shells.
In this post, a brief introduction on the following topics; Appliance shell, File system and the Services.

Read the rest of this entry »


vCSA how to disable IPv6?

18/04/2017

For me it was already a common practice to disable IPv6 on ESXi hosts, but until recently I did not realize that vCenter Server can also benefit from it. For vCenter Server on Windows, you reconfigure the Windows network configuration. But how do you disable IPv6 on the vCSA?

I recently found that a vCSA 6.0 has at least three options to reconfigure the network settings. But only one option enables you to disable IPv6.

Using a web browser you can log in to the vCSA Web console by entering URL: https: //<vCSA hostname or IP address>:5480

From there go to Networking, under Networking Interfaces, choose Edit to open the “Edit IP Configuration” window. Here you can configure IPv4 and IPv6 and disable IPv4, but no option to disable IPv6.

Fig.1

Read the rest of this entry »


vCSA default shell is BASH

14/04/2017

A quick post about a little caveat while working in the vCenter Server Appliance (vCSA) shells. Yes correctly, shells in plural. The vCSA is bundled with at least two different shells:

  • Appliance Shell (default)
  • BASH shell

The “Appliance shell” is the default shell. After you log in to the vCSA, it will present the following well known screen.
Fig 1.

The appliance shell can be used for updating the vCSA, using the software-packages command and has some other use cases. From here you can enable the BASH shell as shown in the Fig 1. for the duration of your session with the following commands:

# shell.set --en -s /bin/bash root
# chsh -s /bin/bash root

You can also set the BASH shell as the default shell by performing the following command. Make sure, you first enable the BASH shell as shown above:

# chsh -s /bin/bash root

For the change to take effect, log out and log in again. Now you will directly enter the BASH shell.

But while working in the BASH shell, you need to temporarily switch to the Appliance shell?
In that case, provide the following command:

# appliancesh

That’s it. A shell is nothing more or less than an executable; the “Appliance shell” is no exception and can be found as /bin/appliancesh.

For more information, see: VMware KB “Toggling the vCenter Server Appliance 6.x default shell (2100508)


Best way to create vCSA support bundles

09/04/2017

In general, during contact with a Customer Support team, whether being a Hardware vendor (Servers, Storage) or a Software vendor, the likelihood that you will be asked to upload some log files for further investigation is significantly.

In case of VMware vSphere, you will have multiple options for collecting a support bundle; to name a few:

  • Windows vSphere Client
  • vSphere Web Client
  • PowerShell scripts

See also VMware KB “Collecting diagnostic information for VMware vCenter Server 4.x, 5.x and 6.x (1011641)” for a complete overview.

Using some of these methods is very convenient; however there is a little caveat, when the vCenter Server is a vCenter Server Appliance (vCSA) 6.x.

Under certain conditions, a vCSA might contain core dump files. When requested to create a support bundle these core dump files will be added to the support bundle, together with the log files. The issue that may arise is that the location where the support bundle will be created (partition /storage/log) has a fixed size and possibly is too small.

Is that is the case, the creation of the support bundle will halt with error “Cannot create a diagnostic bundle” and the desired support bundle will not be created.

VMware recommendation is to create and download a support bundle using the web browser. To do so enter the following URL:
https ://<VCSA Hostname or IP address>/appliance/support-bundle

Fig 1

After providing the credentials, the support-bundle (filename is: vm-support.tgz) will start downloading. The progress of the process will be shown in the browser.

Using this method, the files will be directly downloaded to your local computer, instead of being prepared on the vCSA.

As always, I thank you for reading.