Key performance indicators to determine that an organisation has an effective, efficient, result

oriented TEM function or is close to having one, based on best practise and ITIL/ISO standards

should include the following;

 

• An agreed IT Environments Management strategy or approach is in place, catering for immediate and future requirements. All the relevant stake holders are consulted in establishing the strategy and they fully endorse and approve of the strategy.

 

• An agreed Test Environments Management Process cutting across participation/contributing to the business case of any given project (cost estimates etc) from an IT Environments Management perspective. Provisioning/building through to booking, scheduling, managing, supporting, upgrading, maintaining, re-using test, development and pre-production environments in an organisation. The process could also include  user access authorisation and password management, to ensure that there is no unauthorised access to an environment, to avoid compromising the integrity of any testing or development going on in the environment. The process should be defined in accordance with ITIL/ISO standards and be integrated as much as possible with other service management functions such as Problem, Incident, Change and Capacity management etc.

 

• A Test Environments Management tools such as Omnium Lite, needs to be implemented to cater for environment booking, scheduling, managing, forward planning, reporting and serve as a CMDB for test, development and pre-production environments. Reporting needs to capture trends and distinct patterns of environments usage, across given periods of time such as 6 or 12 months, all such information gathered, will assist immensely in future planning and fore casting.

 

• Between the TEM strategy and process, the costing or charging model to pay for IT environments management related services to include provisioning, maintenance, support etc needs to be agreed and also captured. The two most commonly used models are as follows; (1) Charged as a service directly to projects/programs - test environments as a service (2) Charged to a cost centre, as an already budgeted cost, that is a part of the overall IT budget for that financial year, likely to fall under service management if the organisation is following ITIL precepts. There are obviously varying implications, in adopting either the former or the latter, from a financial management and tax planning point of view.

 

• Frequent governance, test environments management working group meetings with representatives from all active projects/programs, testing team, support teams, service management teams and other stake holders in attendance. A weekly/fortnightly meeting depending on how many projects are currently active or how many environments are in use should suffice.

 

• An annual review of the IT Environments Management process by the user community and stakeholders is also quite necessary.

 

• External audit of the IT Environments Process to confirm that the process is still fit for purpose incorporates best practise and adheres to ITL/ISO standards, should be conducted on an annual basis.

 

• An experienced Test Environments Manager needs to be appointed, who has (apart from the relevant IT skills and experience) significant experience of stake holder management, negotiation, influencing at all levels, trouble shooting and fire fighting skills. The IT Environments manager must be able get to grips very quickly, with any underlying issue or problem impeding testing or development work in an environment and swiftly move to resolve the issue either by matrix managing the relevant teams or resolved directly by members of his IT Environments Management team. The IT Environments Manager must manage and own the resolution of any IT Environments related problems or issues, (or delegate this responsibility to a member of his team), from when it is reported through to when it is resolved.

 

• Raise the visibility of TEM as a function across the whole IT enterprise, ensuring that all the users and stakeholders clearly understand the IT Environments Management process, through workshops, presentations, working group meetings, up to date and relevant information published on the TEM share point team sites (or other similar tools).

 

Other more operational but direct indicators/measurement of how successful (or not) a TEM

function is within an organisation should include;

 

• Adequate test and development environments available for all planned Projects/Programs.

 

• At least 95 to 99% uptime for all test, development and pre-production environments.

 

• Adequate and reasonable turnaround of any technical problems, at least within 24 hours or one business day.

 

• Publish at least weekly, up to date information on IT environments usage, highlighting what environments are in use and for what, which ones are available, connectivity/integration between different environments, data sets available within the environments, code level and all other relevant historical information if necessary.