QC2-PERF

This is the first account of performance monitoring for Qc2

Strategy

The algorithm “StressTester” (https://svn.met.no/viewvc/kvoss/kvQc2/trunk/src/algorithms/StressTester.cc) performs analysis on nine months of data, reads from the database, performs spatial interpolations and writes data to the future, while also pushing interpolation output to a netcdf file. The algorithm is run in isolation and then with kvQabased (Qc1 algorithms). The %MEM and %CPU use is monitored.

The following images just give screenshots of the performance monitoring tools (simple use of top and also vmstat and ps commands piping out to a file (the file is attached below)).

SUMMARY: While the system resources are used intensively by both processes, both processes continue to run and perform adequately. Note, the use of Qc2 in this high stress mode is not typical (current thinking at least).

This is not a sophisticated analysis of performance but a first look that can be built on if required.

The tests were performed on dev-vm116.

Observations

Key

Only kvQabased

Only kvqc2

Running together: typical performance

kvqc2 briefly noses ahead

For reference Log File

Simple script employed:

#!/bin/bash
Xtime=$(date +%s)
Xdate=$(date)
Xvm=$(vmstat)
vmstat
XkvQAB=$(ps aux | grep kvQabased)
Xkvqc2=$(ps aux | grep kvqc2)
echo $XkvQAB
echo $Xkvqc2
while true
do
CPU=$(ps aux| head -1 | awk '{print $3}')
MEM=$(ps aux| head -1 | awk '{print $4}')
kvqc2CPU=$(ps aux| grep kvqc2 | grep -v grep | awk '{print $3}')
kvqc2MEM=$(ps aux| grep kvqc2 | grep -v grep | awk '{print $4}')
QACPU=$(ps aux| grep kvQabased | grep -v grep | awk '{print $3}')
QAMEM=$(ps aux| grep kvQabased | grep -v grep | awk '{print $4}')
Xtime=$(date +%s)
Xdate=$(date)
Xvm=$(vmstat | sed 's/[a-z]*//g' | sed 's/-*//g')
echo $Xtime $Xdate $Xvm %CPU\/%MEM kvqc2: $kvqc2CPU \/ $kvqc2MEM kvQabased: $QACPU \/ $QAMEM
sleep 1
done