Copland

Copland was a project at Apple Computer to create an updated version of the Macintosh operating system. It was to have introduced protected memory, multitasking and a number of new underlying operating system features, yet still be compatible with the vast majority of existing Mac software.

Begun in 1994, it was abandoned in August of 1996 in favor of buying a working third-party solution NeXTSTEP, which eventually became Mac OS X. Starting around 1995, the system was also commonly referred to as System 8 or Mac OS 8, although this name was later re-used for an unrelated operating system, Mac OS 8.

Background

In early 1988, managers at Apple had a meeting to plan the future course of Mac OS development. Ideas were written on index cards; features that seemed simple enough to implement in the short term (like adding color to the user interface) were written on blue cards, while more advanced ideas (like an object-oriented file system) were written on pink cards.^[1] Development of the ideas contained on both sets of cards was to proceed in parallel, and the two projects were known simply as "blue" and "pink".^[2] Apple intended to have the "blue' team (which came to call themselves the "Blue Meanies" after characters in Yellow Submarine)^[3] release an updated version of the existing Macintosh operating system in the 1990–1991 timeframe, and the "pink" team to release an entirely new OS around 1993.

The "blue" team delivered what became known as System 7 on 13 May 1991, but the "pink" team suffered from second-system effect and continued to slip its release into the indefinite future. Some of the reason for this can be traced to problems that would become widespread at Apple as time went on; as "pink" became delayed, engineers on the project jumped ship to work on "blue" instead,^[4] leaving the "pink" team constantly struggling for staffing. Management ignored these sorts of technical development issues, leading to continual problems with delivering working products. In the case of "pink", development eventually slowed to the point that the project was moribund, and Apple semi-abandoned it by spinning it off to form Taligent.^[5] Apple employees made T-shirts graphically displaying that the result would be an IBM-only project.^[6]

This left Apple with no ongoing modern operating system development effort. Originally intended to support a single user running a single application at a time on a non-networked monochrome machine with a single floppy disk drive for storage and printing to a dot matrix printer, the existing Mac OS had evolved gracefully into System 7, but simply did not scale well to the increasing demands of users - it was time for radical changes.^[7] In particular the architecture of QuickDraw made it very difficult to introduce multitasking into the system, the file system was inefficient for devices with larger storage capacities, and networking support was based almost entirely on a proprietary protocol in an era where TCP/IP was already becoming a de-facto standard. Even the lauded user interface, which seemed so elegant when controlling a small screened, floppy disk-based machine, began to be overwhelmed by gigabyte-capacity hard disks and large, high resolution color monitors.

Several attempts were made by various teams to address these issues with an updated operating system, or at least parts of one, but they ran afoul of internal politics and turf wars. John Sculley, Apple's CEO during this period, largely ignored the engineering departments while he concentrated on sales and marketing.

Design

With System 7.5 released in autumn 1994, Apple management decided that the decade-old Macintosh operating system had run its course. The system not only lacked a number of features expected from a modern OS, but was growing increasingly unstable due to the lack of protected memory. The number of crashes suffered by users was increasing all the time, partly due to the use of third party CDEVs and INITs to patch in features missing from the OS, and it was becoming a real concern. An entirely new operating system with more advanced features would be needed for the platform to compete with upcoming releases of Microsoft Windows.

As System 7.5 was code-named "Mozart", the next-generation operating system that was intended to address the looming architectural issues was dubbed "Copland" after composer Aaron Copland. Copland was designed to run the Mac OS on top of a microkernel named Nukernel, which would handle basic tasks such as application startup and memory management, leaving all other tasks to a series of semi-special programs known as servers. For instance, networking and file services would not be provided by the kernel itself, but by servers which would be sent requests though interapplication communications. Copland consisted of the combination of Nukernel, various servers, and a suite of application support libraries to provide the well-known Macintosh programming interface.^[7]

Application services would be offered through a single program known as the "Cooperative Macintosh Toolbox environment",^[7] but universally known as the blue box. The blue box encapsulated an existing System 7 operating system inside a single process and address space. Mac programs would run inside the blue box much as they did under System 7, as 'co-operative tasks' that used the non-re-entrant Toolbox calls. A worst-case scenario was that an application in the blue box would crash, taking down the entire box with it. This would not result in the system as a whole going down, however, and the blue box could be restarted.

Copland runtime architecture. The purple boxes show threads of control, while the heavy lines show different memory partitions. In the upper left is the blue box, running a number of System 7 applications (blue) and the toolbox code supporting them (green). Two “headless” applications are also running in their own spaces, providing file and web services. This diagram, based on one from Apple, shows the operating system servers running in the same memory space as the kernel, indicating co-location.

New applications, those written with Copland in mind, would be able to directly communicate with the new servers and thereby gain many advantages in terms of performance and scalability. They could also communicate with the kernel to “spin off” separate applications or threads, which would run as separate processes in protected memory, as in most modern operating systems. However, these separate applications could not use non-re-entrant calls like QuickDraw, and thus could have no user interface. Apple suggested that larger programs could place their user interface in a normal Macintosh application, which would then start the "work threads" externally. Doing so was not easy, however, as multithreaded code is difficult to write and debug.^[7]

Another key feature of Copland was that it would be completely PowerPC "native". System 7 had been ported to the PowerPC (PPC) with great success; large portions of the system ran in PPC code, including both high-level functionality, such as the majority of the user interface "toolbox" managers, and low-level functionality, such as interrupt management. However there was enough 68k code left in the system, and especially user applications, that the operating system had to map certain bits of data between the two environments.

Most important among these was the sixteen-level interrupt handlers of the Motorola 68000 series CPUs that were widely used in the Mac OS. These could not be easily removed during the transition to the PPC, so they were mapped from the 16-level system of the 68k to the simpler 2-level system of the PPC. This required an expensive call into the OS to translate them in order to call the correct routines in the OS. This was true even if the code on either side of the interrupt was PPC native, which was increasingly common as more and more of the OS and various applications moved to new compilers. Removing this translation step would allow Copland-native applications to run much faster, as much as 50%, with no special effort on the part of the developers.^[7]

The challenge in Copland would be getting all of this functionality to fit into an ordinary Mac. System 7.5 already used up about 2.5 megabytes of RAM, and at the time this was a significant portion of all the RAM available in most machines. Copland would be running what was essentially a complete copy of System 7.5 and an entirely separate operating system running “under” it as well. Copland therefore was to use a Mach-inspired memory management system and rely extensively upon shared libraries, with the goal being for Copland to be only some 50% larger than 7.5.

There were also a number of features that were not a part of the basic Copland system. Notably, Copland did not directly supportmultithreading in the original Mac OS libraries, although programs could be written that directly interacted with the kernel to get these services (the server processes made use of this, for instance). Additionally, Nukernel did not support symmetric multiprocessing, meaning it could not natively utilize more than one processor on a multi-processor system. Both of these features were expected of a modern operating system, but for Copland it was felt that it was more important to get the new API out to developers so they could start moving their applications over to the new system. By the time these capabilities were ready, a good portion of the application base should already be running on the platform.

The plan was that, over time, more and more of the existing non-re-entrant code would be ported to new libraries or servers. Eventually it would become possible that new applications written to these interfaces would be able to be run completely separately as their own tasks. Additionally these applications would be ready to support multiprocessor systems if and when these became available. The follow-on system supporting these features became known internally as Gershwin.^[8] Gershwin was supposed to be a true multi-tasking operating system^[9] and feature a reentrant implementation of the Macintosh Toolbox.^[10]

Development

The Copland project was first announced in March 1995.^[11] Parts of Copland, most notably an early version of the new file system, were demonstrated at Apple's Worldwide Developers Conference in May 1995. Apple also promised that a beta release of Copland would be ready by the end of the year, for full release in early 1996.^[11][12] Gershwin would follow the next year.^[13] Throughout the year, Apple released a number of mock-ups to various magazines showing what the new system would look like, and commented continually that the company was fully committed to this project. By the end of the year, however, the developer release was nowhere in sight.^[12]

As had happened in the past during the development of Blue/Pink, developers within Apple soon started abandoning their own projects in order to work on the new system. Middle management and project leaders fought back, by putting their own project into Copland in an attempt to ensure it could not be canceled and their employees removed to work on Copland.^[14] This process took on momentum over the next year.

Copland's "open file" dialog box. Note the preview area on the right. The "stacked folders" area on the left was intended to provide a visual path to the current selection, but this was later abandoned as being too complex. The user is currently using a "favorite" location.

Soon the project looked less like a new operating system and more like a huge collection of new technologies; QuickDraw GX, SOM and OpenDoc became core components of the system, while completely unrelated technologies like a new file management dialog box (the "open dialog") and "themes" support appeared as well. The feature list grew much faster than the features could be completed, a classic case of creeping featuritis.^[11] An industry executive noted that "The game is to cut it down to the three or four most compelling features as opposed to having hundreds of nice-to-haves, I'm not sure that's happening."^[15] As the "package" grew, testing it became increasingly difficult and engineers were commenting as early as 1995 that Apple's announced 1996 release date was hopelessly optimistic: "There's no way in hell Copland ships next year. I just hope it ships in 1997."^[15]

"Anytime they saw something sexy it had to go into the OS," said Jeffrey Tarter, publisher of the software industry newsletter Softletter. "There were little groups all over Apple doing fun things that had no earthly application to Apple's product line." What resulted was a vicious cycle: As the addition of features pushed back deadlines, Apple was compelled to promise still more functions to justify the costly delays. Moreover, this Sisyphean pattern persisted at a time when the company could scarcely afford to miss a step.^[11]

Developer release

At WWDC '96, Apple's new CEO, Gil Amelio, used the keynote to talk almost exclusively about Copland, now known as System 8. He repeatedly stated that it was the only focus of Apple engineering and that it would ship to developers at the end of summer with a full release planned for late fall. Very few, if any, demos of the running system were shown at the conference. Instead, various pieces of the technology and user interface that would go into the package (such as a new file management dialog) were demonstrated. Little of the core system's technology was demonstrated and the new file system that had been shown a year earlier was absent.

There was one way to actually use the new operating system, by signing up for time in the developer labs. This did not go well:

There was a hands-on demo of the current state of OS 8. There were tantalizing glimpses of the goodies to come, but the overall experience was awful. It does not yet support text editing, so you couldn’t actually do anything except open and view documents (any dialog field that needed something typed into it was blank and dead). Also, it was incredibly fragile and crashed repeatedly, often corrupting system files on the disk in the process. The demo staff reformatted and rebuilt the hard disks at regular intervals. It was incredible that they even let us see the beast.^[16]

After a number of people at the show complained about the microkernel's lack of sophistication, notably the lack of symmetric multiprocessing — a feature that would be exceedingly difficult to add on to an already existing system within a few months — Amelio came back on stage and announced that they would be adding that to the feature list.

In August 1996, "Developer Release 0" was sent to a small number of selected partners.^[11] Far from demonstrating improved stability, it often crashed after doing nothing at all, and was completely unusable for development. In October, Apple moved the target delivery date to 'sometime', hinting that it might be 1997. One of the groups most surprised by the announcement was Apple's own hardware team, who had been waiting for Copland to allow the PowerPC to truly shine. Members of Apple's software QA team suggested, jokingly, that given current resources and the number of bugs in the system they could clear the program for shipping some time around 2030.

Cancellation

Later that summer, the situation was no better, and Amelio hired an outsider to solve the problem; he hired Ellen Hancock away from National Semiconductor to take over engineering and get Copland development back on track.^[17]

After a few months on the job, Hancock came to the conclusion that the situation was hopeless; given current development and engineering, she felt Copland would never ship. She suggested that development continue on the existing Mac OS to improve its stability, while looking outside the company for a new operating system.

Apple officially canceled Copland in August 1996.^[13] While the CD envelopes for the developer's release had been printed, the discs themselves had not been mastered. Among the reasons given for the cancellation were the slow pace of development and the many technical problems remaining to be solved. One of the larger technical hurdles for developers was the need for two PowerPC-based computers simply to install and use the operating system. The two computers would be linked by a serial cable while the software was installed using Open Firmware commands.

Following Hancock's plan, development of System 7.5 continued, with a number of technologies originally slated for Copland being incorporated into the base OS. Apple embarked on a buying campaign, acquiring the rights to various third-party system enhancements and integrating them into the OS. The Extensions Manager, hierarchal Apple menu, collapsing windows, the menu bar clock, sticky notes — all were developed outside of Apple. Stability and performance was improved by Mac OS 7.6, which dropped the "System" moniker. Eventually, many features developed for Copland, including the new Finder and support for themes (the default Platinum was the only theme included) were rolled into Mac OS 7.7, which was rebranded as Mac OS 8.

With the return of Steve Jobs, this rebranding also allowed Apple to exploit a legal loophole to terminate third-party manufacturers' licenses to System 7 and effectively shut down the Macintosh clone market.^[18] Later, Mac OS 8.1 finally added the new filesystem and Mac OS 8.6 updated the nanokernel to handle preemptive tasks. Its interface was Multiprocessing Services 2.x and later, but there was still no process separation and the system still used cooperative multitasking between processes. Even a process that was Multiprocessing Services-aware still had a portion that ran in the blue task, a task that also ran all single-threaded programs (and the only task that could run 68k code).

At the same time, Apple pursued Hancock's alternative plan, and Apple started looking to buy a third-party operating system product. After lengthy discussions with Be and rumors of a merger with Sun Microsystems, many were surprised when Apple bought NeXT in December 1996. The project to port OpenStep to the Macintosh platform was named Rhapsody and was eventually released as Mac OS X, which, prior to the Intel release of Mac OS X 10.4 (Tiger), also used the 'blue box' concept in the form of Classic to run applications written for older versions of Mac OS.

A number of features originally seen in Copland demos, including its advanced Find command, built-in Internet browser, and support for video-conferencing, have reappeared in recent releases of Mac OS X as Spotlight, Safari, and iChat AV, respectively, although the implementation and user interface for each feature is completely different.

See also

• Mac OS history

Bibliography

Notes

References

External links

• Mac OS 8 Revealed (Copland): Copland Development Book
• Apple's Copland project: An OS for the common man Article covering Copland development
• "A Time Machine trip to the mid-'90s", MacWorld article with screen snaps from Copland