Let’s say you have an attribute in vCenter called Business Units.  Each VM in vCenter has a value listed, some are assigned to Sales, some to Marketing, so on and so forth…  Now you want to create services in vFoglight for each of these, but you don’t want to have to keep updating things or creating a new component when a new BU spins up.  Fear not!  That is what I have for you today.

Below is some groovy code you can use to do above.  The best way to use this is to create a rule in vFoglight/Foglight, set it as a Simple Rule and fire every 24 hours (even without data). Here is what that looks like:

Next under the Condition you paste in the code below and change the attribute to whatever you want it to look for.

Last but least, here is the code you can use!

## Begin Code ##

attributeName = “Business Unit”;

ts = server.get(“TopologyService”);

def createOrUpdateObject(typeName, propertyValues) {
def type = ts.getType(typeName);
objShell = ts.getObjectShell(type);
propertyValues.each ({propertyName, propertyValue ->
def prop = type.getProperty(propertyName);
if (!prop.isMany()) {
objShell.set(prop, propertyValue);
}
else if (propertyValue instanceof Collection) {
objShell.getList(prop).addAll(propertyValue);
}
else {
objShell.getList(prop).add(propertyValue);
}
});
return ts.mergeData(objShell);
}

// Find all attribute X values
values = [] as Set;
vmType = ts.getType(“VMWVirtualMachine”);
vms = ts.getObjectsOfType(vmType);

vms.each { vm ->
def custAtts = vm.get(“customAttributes”);
custAtts.each { ca->
caName = ca.get(“name”);
if(caName == attributeName)
{
caValue = ca.get(“value”).trim();
if(caValue.length()>0)
{
values.add(caValue);
}
}
}
}

// Create Services named value Y,Z,… attaching a DMC to each
valSvcs = [];
values.each { val->

// Create the service
valSvc = createOrUpdateObject(“FSMService”, [name:attributeName + " "+val]);

// Create the DMC
dmc = createOrUpdateObject(“FSMDynamicManagedComponent”,
[name:val+" container", container:valSvc,
componentQuery:"VMWVirtualMachine where customAttributes.name = '"+attributeName+"' and customAttributes.value = '"+val+"'",
queryConditions:"customAttributes.name = '"+attributeName+"' and customAttributes.value = '"+val+"'",
baseQuery:"/modules[fqId='system:virtualvmw']/queries[fqId='system:virtualvmw.581']“]);

//     Add the dynamic managed component to the service
valSvc = createOrUpdateObject(“FSMService”, [name:attributeName + " "+val, definition:[dmc]]);
valSvcs.add(valSvc);
}

// Create Service named attribute X
fsmAttService = createOrUpdateObject(“FSMService”, [name:(attributeName), definition:valSvcs]);

// Create Category
fsmCategory = createOrUpdateObject(“FSMCategory”, [name:"Attributes", definition:[fsmAttService]]);
## End Code ##

Being that I dabble a lot in programming, and I work for an ISV, I have to say Carter Shanklin’s session “Getting Stoned with Project Onyx” was simple amazing.  I’d heard about Onyx when it came out late in 2009, but after seeing the product in action I am still trying to grasp how awesome it really is.  In short, Onyx is a code generator for VMware.  It sits between your VI Client and vCenter/ESX and when you perform tasks, it will show you in PowerShell what commands are equivalent to what you performed!  For those that don’t want to learn lots of PowerCLI or you want to follow VMware best practice on coding, it’s an invaluable tool.  You can read more about it here.

Many of the other sessions around Best Practices for Performance APIs, PowerCLI and vOrchestrator were also very good.  While PEX is clearly for partners, I hope VMware continues this high level of presentations this year for VMworld.  I’ll be writing up some thoughts on VMware strategy later this weekend, so stay tuned for a bit more.

For example – I have a Windows 2008 VM that I want to restore some data from C:\ProgramData.  In the GUI for File level restore I see this:

As you can see the directory ProgramData is not there, because it is hidden.  So I browse to the vRanger Service account’s directory (Administrator.Demo) and I can see mount from Windows Explorer.

If I open this and go to the same volume I will now see the folder that is hidden.

From here I can simply copy and past the folder or any files that I choose.  I hope this helps any of you that are looking for those hidden files!

Thomas

First, we need to turn on Custom Attribute collection. In vFoglight Pro 6.0, the default collection is to not collect CA’s. To change this, we must navigate to the collector installation path.

• If you have a default installation it is C:\Program Files\Vizioncore\vFoglight\VMware Agent
• Edit the FoglightAgent.WinService.exe.config with your favorite text editor
• Look for IncludeCustomAttributes
• Change the value from False to True
• Save the file and Close
• Restart the vFoglight Collector & Connector services from Windows Services

After those steps are complete you will now get custom attributes to start showing up in vFoglight Pro. Now, to create some services! Let’s start by navigating to the vFoglight Dashboards Services Service Builder.

From here click Add a New Category and give your Category a name, for example “Business Units” and click Create.

Under the Actions for the new Category, select Add.
Click “Create a new service contained by this service” & select Global Service

Since this is for sales I will call the name “Sales” and click Create.

Now that the “Sales” Service is created we need to add VMs to it.
Under the Actions for Sales Service, select Add.
Click “Create a rule to include a group of components”
I need to assign a name for this rule, which I will call “Sales VMs”.
Next I need to select the type of objects I am looking at, in this case Virtual Machines.
Lastly I need to create the rule condition, which is:
name like ‘%’ && $object.customAttributes.name matches ‘Business.Unit’ && $object.customAttributes.value matches ‘Sales’

At this point I could test the rule to see if it returns the objects I want, but since I know the query is correct I will simply click Create and I will have the components added. I can now see that all of the VMs with the Business Unit that equals Sales are added to my Service. At this point any other service related view or action will apply to this new service. This means I can easily report against the SLA, Health and Availability for my Sales Service.

Figure 1 – JVM Memory Usage

Figure 2 – Load Estimator

As you can see in the figures, there is data growth over time and periodic spikes in load.  What is happening to cause the spikes?  Garbage Collection, GC for short, is running on the vFoglight Management Server during those spikes.  Essentially, as the JVM creates new objects, it uses memory to store the objects.  Over time these objects become unused and the system could reclaim that bit of memory space to reuse for other purposes.  That is what is happening during these spikes, GC is finding unused memory, clearing it out and allowing it to be reallocated.  Garbage collection is akin to Memory reclamation in many senses.

Since the load is acceptable for 100 VMs, we will now go ahead and add in approximately 650 additional VMs.  In total we’re now going to be importing 809 objects.  Objects are comprised of: Virtual Machines, ESX Hosts, Resource Pools, Folders, Clusters, Data Centers & vCenters.

Looking at the new load after adding the additional objects, we can observe that the load is maxing out the 2G of memory during GC periods.  As the host is well undersized for a 64-bit application, it will be shutdown and more memory will be added.
Figure 3 – JVM Memory Usage & Load Estimator

After adding memory to the system, which is also highlighted in Figure 3, we can see load is once again reasonable.  Essentially doubling the resources the vFoglight host, the load of the system has significantly dropped and performance has returned to normal.

vFoglight VM – Updated
OS – Windows 2003 R2 x86_64
vCPUs – 2
Memory – 4G (Changed from 2G)
Hard Drives – 3 VMDKs on Raid Group 2 (only VMDKs on the Datastore)
10G C:\ (OS Partition – 64k aligned)
5G D:\ (Swap Partition 4G Fixed size – 64k aligned)
30G F:\ (Application Partition – 64k aligned)
vFoglight 5.2.6 x86_64

In the next post I’ll be talking about how we start to scale to something much bigger.  We’re already at 809 VMs/objects!  Next we will do over 2500 & 5000 VMs/objects on our way up to over 10,000 objects!

For the purpose of the test I’m using the following configurations to start and we will grow it as we add more vCenters and thus more VMs or as performance dictates.

vFoglight VM
OS – Windows 2003 R2 x86_64
vCPUs – 2
Memory – 2G
Hard Drives – 3 VMDKs on Raid Group 2 (only VMDKs on the Datastore)
10G C:\ (OS Partition – 64k aligned)
5G D:\ (Swap Partition 4G Fixed size – 64k aligned)
30G F:\ (Application Partition – 64k aligned)
vFoglight 5.2.6 x86_64

Physical Hardware
Dell T610
OS – vSphere 4.0
CPUs – 2x Intel Nehalem L5520 @ 2.27Ghz (8 Cores + 8 HT = 16 Logical CPUs)
Memory – 32G
Disks – 3 RAID1 Groups
Group1 = 2x 146G 15K RPM SAS
Group1 = 2x 146G 15K RPM SAS
Group3 = 2x 1.5TB 5400 RPM SATA

A little house keeping before we get into some data.  I’ve chosen to go with the x86_64 version of vFoglight, and intentionally I’ve undersized the requirements to allow us to show a single host and how it can scale to large environments.  As with many 64-bit applications, the memory requirements essentially double vs. 32-bit applications as they require a larger foot print right away to handle more memory pages.  The default behavior of a JVM is to allow for utilization of 75% of total system memory.  For example, below you will see figures around the JVM with top at 1.5G for possible use in the JVM when the total system memory is 2G.

After downloading vFoglight 5.2.6, I simply followed the installation process doing a custom install and changing the paths to F: instead of C: as that is where I want to place the vFoglight application.  This is good for many reasons, but one of the main reasons is it allows me to easily see the Disk IO for the entire application.

FIrst up, a single vCenter with 1 ESX host and 30 Virtual Machines.  After configuring the collector to talk to this vCenter, it does the heavy lifting of getting the data over to the vFoglight Management Server.  Below in Figure 1 & Figure 2 you will see the memory required to run the Management Server, collect the metrics and then the load of the FMS as we see from within the application.

Figure 1 – JVM Memory Usage

Figure 2 – Load Estimator

As you can see, the system is easily handling this small load effectively.  This screen is viewable, as well as lots of other important statistics on your vFoglight server under Dashboards -> Foglight -> Diagnostic -> Performance -> Overview.  Next up, I’ll be adding another 70 so VMs, to take us up to around 100 VMs and then how many more VM’s can we add before adding more memory on our way to 1000 VMs and beyond.

Let’s look at the different parts of vFoglight and talk about each one. There are three components that can be installed together or installed separately. They are the: collector, management server and the database.

The collector is just what it sounds like, it’s an agent that collects data remotely from vCenter and forwards that information into the management system. It’s supported on Windows only, since it currently uses .NET to talk to vCenter.

The second component is the management server. Currently, Vizioncore has a download and go version of vFoglight that runs on Windows x86 or x86_64. This is designed to be the typical easy install with Next -> Next -> Done. There are versions of vFoglight that run on Linux (most 2.6 flavors) and Solaris as well! Also, it’s good to note that vFoglight runs on top of a JVM, which creates pages written mostly in AJAX and served out by Apache.

The last component is the database. vFoglight ships with an embedded version of MySql for use. This is sufficient for most implementations, but once you get into thousands or tens of thousands of VMs, it can be advisable to break the database off. Which is why vFoglight supports external versions of MySql, Oracle, and new with the 6.0 release SQL Server 2005+.

So… all of the components and applications used inside of vFoglight are supported in a VM and there are plenty of success stories from VMware customers & Vizioncore customers that speak to how well each one runs in a VM. vFoglight is no different in that respect. If you plan properly, as with any project, it will run just as well as on a physical host.

For a quick example, I am currently running a test environment with a customer with about 8100+ VM’s being pushed into a single vFoglight VM running SLES 10.2 and the embedded MySql database.

Up next on my plate is to detail out some of these configurations and what some of the largest VMware customers are doing with vFoglight to monitor their virtual & physical infrastructure.

So the first thing to consider is how you wish to model your environment.  This can be the simplest and the hardest part.  If you go ask Gartner, IDC or one of the many large research firms, they will tell you there are lots of ways to model capacity.  You can establish models based on usage, peak performance and a wide variety of other factors.  What I find is most companies are interested in a very simple model to base capacity on.  This is why I use a model I call “Unit” or “Slice” based capacity.

What is a unit or a slice?  It’s whatever you want it to be a unit of measurement for.  So what should I make a slice be?  Only you can answer that, but generally defined a slice as 1 vCPU and/or 1G of memory.  So 2 slices is 2 vCPUs/2G of memory and so on.  With this most customers start to grasp how easier this is compared to counting everything.  The next question is what about storage and networking?  The simple answer is you cannot define capacity for CPU/Memory the same way you do storage.  So storage can have slices as well, but storage is something that needs to be looked at a little differently.

So back to the slices… once you determine your slice model, you get to figure out how many you have.  You can start by looking in vCenter or your ESX hosts and look at all the VMs.  Manually add up the slices based on the total number of slices in use.

For example:
1 Slice = 1 vCPU
1 Core = 4 Slices possible
Host 1 has 8 cores

This would give then yield some results like – ESX Host 1 you have 20 VMS all with 1 CPU each, you would have 20 slices in use, and you have 12 more possible to use.  This is very simple vs. saying you are at 63% CPU utilization and you can add X, Y, Z workloads to use.

The alternative, and more accurate version & more complex, is to look at performance workloads and model those for capacity.  This is where you say I have the same 20 VMs, but you need to look at past performance, say over 30 days or ALL time.  Then you build models to predict future behavior and also to create a model workload based on each.  Along these lines you would have an Exchange Server model, SQL Server model, etc… So you can then say what if I had the Exchange model + 5 SQL Server models + a set of other VMs.   What would that look like now and would I have room in the future to do this?

The math on all of that is extremely complex and doing it by hand is a tedious task.  This is one of the reasons we have a new Capacity Planning & Analytics component in vFoglight Pro 5.2.6.  With this we can take those workloads, define a target host and look to model workloads at a VM to Host level.  This gives a more accurate picture then the Slice model, without forcing you do to all the math.  I’m very excited by this and some of the other things you can do around both models that are both canned in the product and easily created via a custom dashboard yourself.

My next post, I will be talking a little bit more around each model and some interesting ways you can forecast future growth with even more clarity.  Please feel free to drop me an email with suggestions or improvements you would like to see.

In most cases this causes us to look for tools that can develop simple models, with virtualization being put into those models. Recently I spoke with several SMB & Enterprise level users and they told me that they’re essentially looking for a way to take a ‘slice’, and to know how many of those slices they still have and when they’ll run out. This approach assumes 100% utilization of the resources, and that’s just not realistic.

The problem is that people are still thinking about capacity from a physical hardware perspective where you get X resources because it may need X resources. With all the benefits that come with virtualization it’s time for us as a community to develop new models that can adapt to each environment, and better yet accurately predict performance and capacity bottlenecks. This is why I’m a strong proponent of looking at each individual workload and correlating those workloads vs. ‘servers or vms’.

A virtual machine is nothing more than a shim, a place to wedge an application that performs some task. If you start to look at the VM as nothing more than an application workload, that will model far better to true capacity planning for a specific resource. Does 1 vCPU for a file server equal 1 vCPU for a SQL server? Certainly not, but the general capacity models don’t take that into account. Likewise disk utilization on a file server is going to be far higher than that of a print server. So looking at the application stacks or workloads is required if you want any level of clarity. In the end this will lead to better utilization of resources, and best of all it can help predict when new hardware will be required for net-new capacity with a relatively high level of accuracy.