Scalability of Office Suites on Terminal Servers

Posted by Benny Tritsch on May 10, 2007

[Introduction] [Methodology] [Performing the Test] [Measurements] [Results] [Conclusions]
[Appendix 1: Configuration Details] [Appendix 2: Step-by-Step Description]

Results

The logon sequences ran synchronously on all servers, a fact which was guaranteed by the vRD tool on the load generators. The results were therefore easily comparable.

On the 32-bit server with Microsoft Office 2003, it was possible for all 120 users to log on and to start Office applications including Word, Excel, and PowerPoint with the accompanying documentation. This resulted in a total of 1,238 processes, whereby 120 instances each of Word, Excel, and PowerPoint contributed a total of 360 processes.

 

 

However, on the 32-bit server with Microsoft Office 2007 only 100 users were able to log on. After that point, the system was saturated and would not permit any further user logons. 97 Word, 97 Excel, and 95 PowerPoint processes were started. At the end of the test, a maximum of 916 processes and 7,256 threads were running on the system.

Similarly, only 100 users were able to log on to the 32-bit server with OpenOffice.org 2.1 before the system became saturated and stopped permitting logons. A total of 928 processes on OpenOffice.org 2.1 with Writer, Calc, and Impress could be observed. All OpenOffice applications were activated within the Soffice.bin runtime environment and thus did not form their own processes. The Soffice.exe start process is additionally necessary per user session. 99 instances of both processes, Soffice.exe and Soffice.bin, were observed on the server with OpenOffice.

On the 64-bit servers, all 120 users could log on with Office 2003 as well as OpenOffice.org, whereas roughly 115 users could log on with Office 2007. The remaining observations relating to logon mechanisms can be transferred from the 32-bit world to the 64-bit environment.

 

 

The differences in saturation behavior exhibited by the 32-bit servers on the respective office suites are even more pronounced when considering the available memory on the servers over the length of the test period. At the beginning of the 32-bit test runs, roughly 3.3 GB free memory was available. Roughly 100 users could log on to the server with Microsoft Office 2003 and 80 users on Microsoft Office 2007 before the memory started running low, so giving rise to obvious paging activity; the same happened on the server with OpenOffice.org 2.1, but as early as with 50 users.

 

 

For the 64-bit servers that had the double amount of physical memory with 8 GB, this behavior was only observed later. The available memory for OpenOffice.org ran low with 90 users, with the same occurring with 110 users on Microsoft Office 2007. The memory with Office 2003 was sufficient for no bottlenecks to appear during the entire test run.

 

 

At this point, it was necessary to take a closer look at memory use. The fundamental memory requirements of the applications on the 32-bit servers with Office 2003 were as follows:

• Microsoft Word: The typical working set was 20 MB, the maximum value ran to up to about 26 MB.
• Microsoft Excel: The typical working set amounted to about 6.7 MB, while the maximum value was around 12 MB.
• Microsoft PowerPoint: The typical working set was 1.8 to 2.2 MB, the maximum value was 22 MB.

When the three applications are started in a user session with the corresponding documents, initial memory requirements are 60 MB; this reduces to 30 MB after a relatively short period of time. The first “trimming” of the working set occurs before memory limits are reached.

On the 64-bit system, the Office 2003 applications used the following memory resources: Microsoft Word on average 26-27 MB (27,3 MB initial maximum value), Microsoft Excel on average 13,6 MB (13,6 MB initial maximum value), and Microsoft PowerPoint on average 1.8-2.9 MB (23 MB initial maximum value). The memory requirements per user session amounted to nearly 65 MB at the beginning and dropped to 45 MB with time. The operating-system memory management did not have to start the massive paging at any time.

For the 32-bit server with Microsoft Office 2007, the following memory usage by the applications was observed:

• Microsoft Office Word: The typical working set was 22 MB, the maximum value amounted to roughly 28 MB.
• Microsoft Office Excel: The typical working set was 16 MB, the maximum value amounted to roughly 22 MB.
• Microsoft Office PowerPoint: The typical working set was 4 MB, the maximum value amounted to roughly 38 MB.

Initially, the memory requirement of the three Office 2007 applications amounted to just over 80 MB, although this requirement dropped to 40 MB per user session in a short time.

For the 64-bit system, the Office 2007 applications used the following memory resources: Microsoft Word on average 26 MB, which also represented an initial maximum value, Microsoft Excel on average 23,8 MB (24 MB initial maximum value), and Microsoft PowerPoint on average 3,8-7,0 MB (45 MB initial maximum value). The initial memory requirement therefore amounted to nearly 95 MB per user session and was then reduced to 55 MB. In addition, the Splwow64 process contributed an extra 5 MB for each user session whereby the runtime environment supported 32-bit processes on the 64-bit platforms. The 64-bit platform with 8-GB RAM did not reach its limits with this memory requirement even after 100 users had signed on.

For OpenOffice.org, the situation at the processor level was a bit different, especially because the individual applications were not directly distinguishable by their memory footprint. All applications were started in the OpenOffice runtime environment Soffice.bin. The following values resulted on the 32-bit platform:

• Soffice.bin: The typical working set amounted to 30 MB, although the maximum value reached 84 MB.
• Soffice.exe: The typical working set was only 200 kB and the accompanying maximum value was 2 MB.

The maximum value of the three OpenOffice applications along with their accompanying documents therefore amounted to nearly 90 MB per user session. Obviously an operating system has much more difficultly to manage the working sets for application components which are encapsulated in a runtime environment.

With the 64-bit edition of Windows Server 2003, the typical working set of Soffice.bin was 78 MB; the maximum value amounted to 87 MB. Soffice.exe merely added a working set of 200 kB and a maximum value of just 3 MB.

 

 

The graph quite clearly indicates that the Windows operating system can manage its memory more efficiently on the 32-bit server with Office 2003 and Office 2007. “Trimming” of the working set when critical limits are reached, evidenced by the characteristic sawtooth pattern, demonstrates the optimization of memory as used by each individual process. The optimization of memory size results in freed memory, which can be used by new processes. The typical quantity of memory that can be freed on servers with Microsoft Office amounts to about 1 GB per sawtooth.

On the 32-bit server with OpenOffice, the amount of working sets for all processes rises faster than with Microsoft Office; the saturation point is reached with just 50 users. The memory must be massively paged to the hard drive, which leads to additional system activities. For the 32-bit server with Microsoft Office 2003, similar behavior can be observed with more than 100 user sessions, and for Office 2007, similar behavior can be observed with more than 80 user sessions. When the number of 110 users is reached—which is no problem for the system on Office 2003—the 32-bit server with OpenOffice, and even with Office 2007, has to allocate all of its resources and cannot permit any additional user logons anymore.

This pattern can also be observed in the use of paging files. While the server with OpenOffice hardly uses the paging file at all initially, Microsoft Office permanently pages data to this file.

 

 

The evaluation of the 64-bit systems also displays a clear working set increase over all processes. Due to the higher amount of physical memory, none of the typical sawtooth patterns can be viewed for the massive trimming of the working set by memory management. Only with OpenOffice.org does the system attempt to optimize memory usage in smaller intervals.

 

 

Access to the page file follows a similar pattern to the 32-bit systems. The memory management of the server with OpenOffice first begins to actively use the page file when saturation limits are reached. The server with Microsoft Office on the other hand stores process data from the very beginning, thus raising the requirements for the size of the page file. This in turn can trigger saturation effects.

 

 

Upon closer inspection of hard disk activity, the different resource requirements are confirmed. As seen above, increased access to the hard drive, which causes longer waiting times, begins on the 32-bit server with OpenOffice.org 2.1 with 50 or more users due to the draw on free memory. For the 32-bit server with Microsoft Office 2007 and Office 2003, similar patterns first develop with roughly 80 and 100 users, respectively.

The onset of such concentrated memory paging often leads to a clearly noticeable decline to the system reaction within the individual user sessions on a terminal server. This behavior can be observed in tests as well. The logon times for the sessions will become longer after the fiftieth user has logged on to the system with OpenOffice.org. For users, interaction with the system will become markedly slower as of this moment.

 

 

Marked increases to queue lengths will only be observed on OpenOffice.org because Microsoft Office’s permanent memory optimization of the individual related processes does not necessitate sudden, massive paging actions on behalf of memory management.

 

 

Along with memory load, differences in processor loads are also apparent. On average, the 32-bit server with OpenOffice.org 2.1 displays a higher load than the comparable 32-bit server with Office 2003 or Office 2007. OpenOffice.org generates a mean processor load of 20%, whereas the figure for servers with Microsoft Office is around 10%. Large spikes to CPU load are also obvious once the physical memory is full, requiring massive paging activities.

 

 

The 64-bit edition of Windows Server 2003 appears similar to the 32-bit platform in terms of processor load on the server with various office suites.

 

 

Next