Situation: Remotely Managing Large Computer Centers in a timely and cost effective manner

Read these questions and see if you are looking for the answers that MO provides:

Should Operations Staff lose sleep and upset significant others by having to travel in late at night to reboot or re-power a down system?
Should the need for physical access by operations and system admin staff to the compute center be eliminated to improve data center security?
Or Should you reduce the need for physical access by operations and system admin staff to the compute center to improve security?
Should operations staff have access to the latest technologies such as Intel’s Active Management technology?
Should system administrators have access to KVMs and serial ports of many different vendors from one unified command console?
Should Users wait to have a system rebooted or re-powered or should it be done automatically with notification?
Should Security have a log of each session on the serial command ports?
Should Operations and Firewall Administrators know what rules were changed, by who and when without compromising the internal Network’s Security?
Should third party vendors be provided with a secure, fully logged access to your system rather than via labor intensive dialup modem?
 Should users be automatically logged onto a HP ILO or LOM to begin their work or to power down a system?
Should your operations staff leave systems’ critical command and control ports connected to a terminal server and not know who is connected to the terminal server or the system?
Should  more than one person have access to the system’s command port at the same time?
Should your data center policy require system admin or operations staff to continually engage compute room technicians or facilities staff to accomplish the task of rebooting or re-powering a device?
Should critical info be lost that is coming to a serial port of a server system including messages that only come to the console port?
Should time should be lost in not having this critical info acted upon immediately by an automated response such as reporting the problem to the vendor/responsible staff person(s) or take planned action?
Should space be taken up by monitors in a data center?
Should you make it easier to allow outside vendors to remotely access the systems they support?
Should you provide vendors with same remote access you have rather than requiring them to actually enter the compute center or be the man in van who always has to go on site and then go offsite to get the replacement part?
Should  operations staff have the ability to turn off power for more than one system at a time?
Should the operations staff be required to memorize what port is connected to what server on what terminal server?
 Should operations accessing a system in trouble be able to view the last bit of activity on the system’s serial command port?
Should operator have to remember what they did to fix a problem or have a log of what they did to fix it?
Should operations be able to access more than one system at a time from a single command window to reduce the time to complete repetitive tasks to same type devices?
Should data center managers have the ability to define who gets to send breaks to systems that need to be rebooted?
Should operations staff know what signals are actually active on system’s console port to determine whether it is actually turned off or just in need of reboot?
Should it be important to know the immediate availability of the server, network device or storage equipment?
Should it be important to automated response to problem?
Should it be important to get an immediate health of the system snap shot?
Should operations staff be able to access a system whose login access has been maliciously tampered with?
Should facilities and operations staff  have the same view of the same devices they both have responsibility for?

MO is your answer if these questions are important!

Trouble Free Large Scale Console Manager Deployed

The Business Challenge: Provide operations staff from anywhere in the world secure ( encrypted ) and instant access. To perform system management tasks and monitor console input and output. The Business Result: Product Administration is done by a staff of one for even the largest of installations. The product provided many benefits available from BMC Software for no additional cost and very little system admin overhead. The MO installation was nearly trouble free and within 15 minutes I was able to start the MO application GUI to start adding consoles. It quickly became apparent that MO is more than just a simple console manager. MO is a comprehensive system management product that provides log file management, event management and notification, and terminal server management. Along with this comprehensiveness comes complexity. The Story: Background: MARS had approximately 75 consoles (mostly UNIX systems and network gear such as Cisco routers and switches) with about 60 of these consoles connected to a couple of Xylogics Annex3 terminal servers. There was no console management solution in place at the time and staff would simply telnet directly to the terminal server port in order to connect to particular consoles. Obviously this was an unworkable solution and I set out to find and deploy a true console management solution. (I had used DEC's Polycenter Console Manager (PCM) at two previous employers.) At the same time, I continued to add console connections to existing and new terminal servers. Before beginning my search, I identified and ranked the desired attributes: Console Management - The console manager "manages" which terminal server port is which console - no remembering which port is which Console Control - Easily manage the console (e.g. review the last N lines, send a "break" character, etc.) Console Logging - Useful for diagnosing system crashes or for auditing console access Read-Only Console Connection - Provides the ability to watch over another user's shoulder Security Mechanism - Only permit authorized users to connect to a console Event Notification - Provide notification (e.g. email, pager) of specific console text My initial research focused on commercial console management solutions including MO (Manage Operations) from Manage Operations and ControlTower from Aurora Technologies. Unfortunately, while these commercial products provided rich and robust console management features, they were also prohibitively expensive for my environment which had grown to 178 managed consoles. Since I consider Console Management a critical production facility, one of my minor concerns was understanding the terms of use of this permanent license and the support options. Happily, Manage Operations provided formal license and support documents that were quickly approved by my management. Once the red tape was worked out, I began planning the deployment of MO across my enterprise. MO Deployment: My current environment is four terminal servers with 178 managed consoles. The MO installation was nearly trouble free and within 15 minutes I was able to start the MO Applications GUI to start adding consoles. It quickly became apparent that MO is more than just a simple console manager. MO is a comprehensive system management product that provides log file management, event management and notification, and terminal server management. Along with this comprehensiveness comes complexity. While Conserver is driven by a fairly simple configuration file, MO is best configured via the Ki Works GUI. (There is a CLI configuration utility, but I've not explored its capabilities yet.) Since my primary goal is console management, I started by creating several console connections. The first step is to define the terminal server(s). This is a simple task that is made easy by predefined configurations for most popular terminal servers (e.g. Xylogics, Cyclades, Lantronix, Cisco, etc). Once a terminal server is defined, adding a console is just a simple. Simply name the console, select the terminal server it is connected to, specify the port number on the console, and select the hardware and operating system type. (The last two options instruct MO how to interact with the console. For instance, how to send a break, what the boot prompt is, etc.) After creating a console, double clicking on the console icon pops up a console window and off you go. There are actually several methods of connecting to a console: Simple Text Connection in a window (i.e. xterm, dtterm) GUI connection in a small graphical app with drop-down menus for issuing common commands such as sending a break, powering off a system, etc. Read-Only Connection in a window Command Line from a shell prompt and, most intriguing: Gang Connect which lets you connect to multiple consoles at once and type the same commands into all connections. Very handy! I continue to discover features and benefits of MO as I use the product. For instance, the MO Event Management module (EVM) includes a large set of preprogrammed console event types. For instance there are over a hundred UNIX-specific events. Some examples include system panic, file system full, etc. At the very least, these preprogrammed event types provide an example of how to create custom, application/environment specific event types.

 

A Global Food and Personal Care Company

Lights Out Without A Torch

The Business Challenge: In a lights out situation, there are always times, when that server just cannot be connected to, when you just don't know what caused the system crash, or you have so many consoles for all the servers that you need a separate room just to house them, and a torch to find your way round the computer room in the dark

The Hardware Solution: HP of course came up with a solution, but this would require a separate control box for each system, and an IP address for each of these control boxes, Of course with the normal HP overhead in costs as well. Business Result: After using MANAGE OPERATIONS Software we were able to control all the servers in the computer room without being near there, completely install a server from scratch, add patches, reboot and find causes of crashes, all using existing resources, and without leaving the comfort of your home.

The Story: We had a large number of HP UX servers and many Windows, plus a few AIX units in a computer room that was intended to be lights out. There were however many times when the auto callout that had been created to fit within service level, the need to re-install servers, or add patches, system hangs or crashes, and the need to power cycle boxes made being in the computer room a requirement. And with support being carried out from a different location, and there not being any operations staff this would mean resources being stretched. With the use of Manage Operations software the requirement for all the above was able to be carried out from remote locations, any hour of the day or night. 

Working closely with MANAGE OPERATIONS were able to place the HP and AIX servers to a links that could be used for whatever was required. This basically placed you at the console, and give you everything that would be required for console level activities. This also provided a log of console reported actions, so any crashes and lead ups to crashes could be seen and actioned on. There are added security functions of read only, and password control access, so you can see and record actions being carried out by 3rd party support. Using power control switches we were also able to reboot servers and communications equipment remotely fixing the problems that surround those outages. This software fixed many requirements and saved much loss of sleep and having to drive to site locations to fix simple problems.  

Large British Utility:  

We have two large compute centres; support staff needed to access these centres remotely. Manage Operations software provided a single tool to access these centre’s servers using multiple vendor’s console servers and KVMs for our support staff located throughout the country.

 Large Global Pharmaceutical Company

 There are multiple R&D sites that require strict regulatory compliance and quick secure access. We have just recently upgraded to the emser-enabled MO software as we replace the old terminal servers with Lantronix consoles. Using MO allows us to access all consoles from a central point (and a peer/backup as well), without having to login to each terminal server individually. The upgraded version does provide the following:

 1. one connection per MO server to Lantronix Terminal Server rather than the current 96 connections per 48 port terminal server

 2. the quick ( 60 second ) recovery time to re-establish connection to ports from power on and Lantronix OS boot compared to making 96 connections which sometimes are not made as the SSHD on the Lantronix cannot handle 48 concurrent request for port which create 96 open network connections when done.

 3. secure login-less access to the command prompt of the Lantronix OS compared to having to ssh or telnet into the Lantronix

 4. IML’s emser maintains access to the Lantronix’s ports which means that multiple MO servers and emstty on the Lantronix itself can have access to the port; there is no need to be restricted or limited to “a one Management Server access to one Lantronix terminal server port”. Multiple Management/MO servers can have access and define the same port; makes redundancy /failover ridiculously simple!

 5. allows for convenient "read-only" console connections which still enables multiple analysts to connect to a console while only one is controlling it (read/write access)