Our custom Redhat Linux system uses fvwm. The primary application on this system is Vmware. A user logs in and has a menu of all the VMs she is allowed to run. We have things pretty locked down, so the user has little control over anything. One thing the user can control is how the VM is stopped when they are done. We have a Fvwm menu item called “Player Preferences” which allows the user to select one of two Vmware options regarding how the VM shall be stopped when the user presses the ‘X’ in the upper right corner of the window manager border. When they select one of these options, we write the appropriate entry into the .vmx file.
These two options are: 1) Powerdown the VM; or 2) Suspend the VM
There are no issues with option 1. However, if the user selects option 2, then the following behavior will occur.
After bringing up and suspending the VM two or three times, the user will then not be able to do anything, meaning all the fvwm menu items don’t respond. Since the user has no way to bring up a shell or anything, and logout, reboot, shutdown don’t work from their menu, they are forced to power down their machine with the power button.
A debug version of the system shows that fvwm is holding nearly 1000 FIFO’s in the users name, and hence the menu items no longer work since no more FIFOs can be opened. Note that these FIFOs are just the ones held by the fvwm command under the users UID. This is seen using the lsof command.
If the user chooses the powerdown option for the VM, none of this happens. In this case, just at login, fvwm will hold about 80 FIFOs; after the VM is launced, fvwm will hold about 200 FIFOs, then when the VM is powered down, we go back to about 80 FIFOs.
In the VM suspend case, the number of FIFO’s continually escalates up to the limit at which the system is unresponsive. In a debug build of the system, I can bring up an xterm with a hot-key combination and kill the fvwm process.
I am just wondering if anyone has any ideas, is there more info I could provide, what things can I do to narrow down the problem, etc.