downtime

The term downtime is used to refer to periods when a system is unavailable. Downtime or outage duration refers to a period of time that a system fails to provide or perform its primary function. Reliability, Availability, Recovery, and Unavailability are related concepts. The expected Unavailability is the percentage of a timespan that a system is unavailable or offline. This is usually a result of the system failing to function because of an unplanned event, or because of routine maintenance.

The term is commonly applied to networks and servers. The common reasons for unplanned outages are system failures (such as a crash) or communications failures (commonly known as network outage).

The term is also commonly applied in industrial environments in relation to failures in industrial production equipment. Some facilities measure the downtime incurred during a work shift, or during a 12 or 24-hour period. Another common practice is to identify each downtime event as having an operational, electrical or mechanical origin.

The opposite of downtime is uptime.

Characteristics

Unplanned downtime may be the result of a software bug, human error, equipment failure, malfunction, high bit error rate, power failure, overload due to exceeding the channel capacity, a cascading failure, etc.

See also: Planned downtime

Telecommunication Outage Classifications

Downtime can be caused by failure in hardware (physical equipment), software (logic controlling equipment), interconnecting equipment (such as cables, facilities, routers,...), Wireless transmission (wireless, microwave, satellite), and/or capacity (system limits).

The failures can occur because of damage, failure, design, procedural (improper use by humans), engineering (how too use and deployment), overload (traffic or system resources stressed beyond designed limits), environment (support systems like power and HVAC), scheduled downtime (outages designed into the system for a purpose such as software upgrades and equipment growth), other (none of the above but known), or unknown.

The failures can be the responsibility of customer/service provider, vender/supplier, utility, government, contractor, end customer, public individual, act of nature, other (none of the above but known), or unknown.

^[1]

Impact

Outages caused by system failures can have a serious impact on the users of computer/network systems, in particular those industries that rely on a nearly 24-hour service:

• medical informatics
• nuclear power and other infrastructure
• banks and other financial institutions
• aeronautics, airlines
• news reporting
• e-commerce and online transaction processing
• persistent online games

Also affected can be the users of an ISP and other customers of a telecommunication network.

Corporations can lose business due to network outage or they may default on a contract, resulting in financial losses.

Those people or organizations that are affected by downtime can be more sensitive to particular aspects:

• some are more affected by the length of an outage - it matters to them how much time it takes to recover from a problem
• others are sensitive to the timing of an outage - outages during peak hours affect them the most

The most demanding users are those that require high availability.

Famous outages

On Mother's Day Sunday, May 8, 1988, a fire broke out in the main switching room of the Hinsdale Central Office of the Illinois Bell telephone company. One of the largest switching systems in the state, the facility processed more than 3.5 million calls each day while serving 38,000 customers, including numerous businesses, hospitals, and Chicago’s O’Hare and Midway Airports.^[2]

Virtually the entire AT&T network of 4ESS toll tandems switches went in and out of service over and over again on Jan. 15, 1990 disrupting long distance service for the entire nation. The problem dissipated by itself when traffic slowed down. A software bug was found.

AT&T lost its frame relay network for 26 hours on April 13 1998^[3]. This affected many thousands of customers, and bank transactions were one casualty. AT&T failed to meet the service level agreement on their contracts with customers and had to refund^[4] 6600 customer accounts, costing millions of dollars.

Service levels

In service level agreements, it is common to mention a percentage value (per month or per year) that is calculated by dividing the sum of all downtimes timespans by the total time of a reference time span (e.g. a month). 0% downtime means that the server was available all the time.

For Internet servers downtimes above 1% per year or worse can be regarded as unacceptable as this means a downtime of more than 3 days per year. For e-commerce and other industrial use any value above 0.1% is usually considered unacceptable.

Response and reduction of impact

It is the duty of the network designer to make sure that a network outage does not happen. When it does happen, a well-designed system will further reduce the effects of an outage by having localized outages which can be detected and fixed as soon as possible.

A process needs to be in place to detect a malfunction - network monitoring - and to restore the network to a working condition - this generally involves a help desk team that can troubleshoot a problem, one composed of trained engineers; a separate help desk team is usually necessary in order to field user input, which can be particularly demanding during a downtime.

A network management system can be used to detect faulty or degrading components prior to customer complaints, with proactive fault rectification.

Risk management techniques can be used to determine the impact of network outages on an organisation and what actions may be required to minimise risk. Risk may be minimised by using reliable components, by performing maintenance, such as upgrades, by using redundant systems or by having a contingency plan or business continuity plan. Technical means can reduce errors with error correcting codes, retransmission, checksums, or diversity scheme.

Planning

A planned outage is the result of a planned activity by the system owner and/or by a service provider. These outages, often scheduled during the maintenance window, can be used to perform tasks including the following:

• Deferred maintenance, e.g., a deferred hardware repair or a deferred restart to clean-up a garbled memory
• Diagnostics to isolate a detected fault
• Hardware fault repair
• Fixing an error or omission in a configuration database or omission in a recent configuration database change
• Fixing an error in application database or an error in a recent application database change
• Software patching/software updates to fix a software fault.

Outages can also be planned as a result of a predictable natural event, such as Sun outage.

Maintenance downtimes have to be carefully scheduled in industries that rely on computer systems. In many cases, system-wide downtimes can be averted using what is called a "rolling upgrade" - the process of incrementally taking down parts of the system for upgrade, without affecting the overall functionality.

Avoidance

For most websites, website monitoring is available. Website monitoring (synthetic or passive) is a service that "monitors" downtime and users on the site.

Other usage

Downtime can also refer to time when human capital or other assets go down. For instance, if employees are in meetings or unable to perform their work due to another constraint, they are down. This can be equally expensive, and can be the result of another asset (i.e. computer/systems) being down. This is also commonly known as "dead time".

See also

• High availability
• Uptime
• Mean down time
• Carrier grade

References