Copland (operating system)

Copland was a project at Apple Computer to create an updated version of the Macintosh operating system. It was to have introduced protected memory, preemptive multitasking and a number of new underlying operating system features, yet still be compatible with existing Mac software. As originally planned, a follow-up release known as Gershwin would add multithreading and other advanced features.

Development began in 1994 and was underway in earnest by 1995, when the system started to be referred to as System 8, and later, Mac OS 8. As the project gathered momentum, a furious round of empire building began. New features began to be added more rapidly than they could be completed, including most of the items originally slated for Gershwin, along with a wide variety of otherwise unrelated projects from within the company. The completion date continued to slip into the future, and several key dates passed with no sign of a release.

In 1996, Apple's newest CEO, Gil Amelio, poached Ellen Hancock from National Semiconductor and put her in charge of engineering in an effort to try to get development back on track. She decided it was best to cancel the project outright and try to find a suitable third-party system to replace it. Development officially ended in August 1996, and after a short search they announced that Apple was buying NeXT in order to use their NeXTSTEP operating system as the basis of a new Mac OS.

Hancock also suggested that Apple should work on improving the existing System 7 while the new system matured. This was released as Mac OS 8 in 1997, and was followed by Mac OS 9 in 1999. The new operating system based on NeXTSTEP shipped in 2001 as Mac OS X.

In 2008, PCWorld magazine named Copland to a list of the biggest project failures in IT history.[1]

Background

Mac OS

Like most personal computer operating systems of the era, much of the original Mac OS consisted of library routines and a terminate and stay resident launcher. When the user launched an application, the program would load and then control of the computer and all its resources would be handed to it. When the application quit, the loader would switch to another pre-defined application, typically the Finder. Only one program was running at one time, and that program was in control of the system.

The additional hardware needed to support a GUI and be able to run multiple programs at once led to much higher priced machines like the Apple Lisa. The Mac was designed from the outset as a single-user, single-tasking machine, which allowed the hardware to be greatly simplified.[2] As a side effect of this single application model, the original Mac developers were able to take advantage of a number of simplifications that allowed great improvements in performance, running even faster than the much more expensive Lisa. But this design also led to several problems for future expansion.

By assuming only one program would be running at the same time, the engineers were able to ignore the concept of reentrancy, the ability for a program (or library) to be stopped at any point, asked to do something else, and then return to the original task. In the case of QuickDraw (QD) for instance, this meant that the system could store state information like the current location of the window or the line style internally, knowing that it would only change under control of the running program. Taking this one step further, the engineers left most of this state inside the application, not QuickDraw, eliminating the need to copy this data between the application and library.

The other main issue was that the early Macs lacked a memory management unit (MMU)s. MMUs provide key functions in modern computers; they map memory addresses from main memory to each application so that the applications believe they are running in a space of their own, they provide memory protection that ensures programs cannot accidentally overwrite other program's memory, and they provide systems to move data from program to program in a safe way using shared memory. Lacking the last of these, shared memory, the API was instead written so that the operating system and application shared all memory, which is what allowed QD to examine the program's memory for settings like the line drawing mode or color.

“ The Macintosh lacks multitasking but tries to fake it, and it insists on a complicated user interface, but leaves much of the work up to the application. These are serious drawbacks, and it is difficult to imagine elegant repairs for them. --Byte, Sept. 1986[3] ”

These limitations meant that supporting more than one program at once would be difficult without rewriting all of this code. Yet doing so would mean that the new system would run unacceptably slow on existing hardware. Instead, Apple adopted a system known as MultiFinder in 1987 that kept the running application in control of the computer as before, but allowed that application to be rapidly switched to another, normally simply by clicking on its window. Programs that were not in the foreground were periodically given short bits of time to run, but as before, the entire process was controlled by the applications, not the operating system.

Because the operating system and applications all shared a single memory space, it was possible for a bug in any one of them to corrupt the operating system and crash the machine. This was not particularly annoying under a single-application model, in which case the application crashed anyway. But under MultiFinder, that crash also crashed all the other running programs as well. And because there was more of them running, the chance of a crash was higher.

Adding greatly to the severity of the problem was the system used to add functionality to the operating system itself, which relied on a patching mechanism known as CDEVs and INITs. Third party developers also made use of this mechanism to add features like screensavers and a hierarchal Apple menu, independently of Apple. Some of these third-party control panels became almost universal, like the popular After Dark screensaver package.[4] Since there was no standard on how to use these patches, it was not uncommon for several of these add-ons, including Apple's own additions to the OS, to use the same patches and interfere with each other, leading to more crashing.

'Pink and Blue'

In March 1988,[N 1] technical middle managers at Apple held an offsite meeting to plan the future course of Mac OS development.[5] Ideas were written on index cards; features that seemed simple enough to implement in the short term (like adding color to the user interface) were written on blue cards, longer-term goals like preemptive multitasking on pink cards, and "far out" ideas like an object-oriented file system on red cards.[6][7] Development of the ideas contained on the blue and pink cards was to proceed in parallel, and at first the two projects were known simply as "blue" and "pink".[8] Apple intended to have the "blue" team (which came to call themselves the "Blue Meanies" after characters in Yellow Submarine)[9] release an updated version of the existing Macintosh operating system in the 1990–1991 timeframe, and the "pink" team to release an entirely new OS around 1993.

The "blue" team delivered what became known as System 7 on May 13, 1991, but the "pink" team suffered from second-system effect and continued to slip its release into the indefinite future. Some of the reason for this can be traced to problems that would become widespread at Apple as time went on; as "pink" became delayed, engineers on the project jumped ship to work on "blue" instead.[10] This left the "pink" team constantly struggling for staffing and suffering from the problems associated with high employee turnover. Management ignored these sorts of technical development issues, leading to continual problems delivering working products.

At the same time, the recently released NeXTSTEP was generating intense interest in the developer world. Features that were originally part of "red" were folded into "pink" and the red project (also known as "Raptor")[11] was eventually cancelled. This problem was also common at Apple during this period; in order to chase the next big thing, middle managers would add new features to their projects with little oversight, leading to enormous problems with feature creep. In the case of "pink", development eventually slowed to the point that the project appeared moribund.

Taligent

On April 12, 1991, Apple CEO John Sculley demoed Pink running on an PS/2 Model 70 to a delegation from IBM in a top-secret mission. The system was not fully functional, but resembled System 7 running on a Pentium PC. IBM proved extremely interested and over the next months the two companies formed an alliance to further development of the system. These efforts became public in early 1992, under the new name "Taligent".[12] At the time, Sculley summed up his concerns with Apple's own ability to ship Pink when he stated "We want to be a major player in the computer industry, not a niche player. The only way to do that is to work with another major player."[13]

Infighting at the new joint company was legendary and the problems with Pink within Apple soon appeared to be minor in comparison.[14] Apple employees made T-shirts graphically displaying their prediction that the result would be an IBM-only project,[15] a prediction that came true on December 19, 1995 when Apple officially pulled out of the project.[16] IBM continued working with Taligent, and eventually released its application development portions under the new name "CommonPoint". This saw little interest and the project disappeared from IBM's catalogs within months.

Business as usual

While Taligent efforts continued, very little work addressing the structure of the original OS was carried out. Several new projects started during this time, notably the Star Trek project which reached demo quality. But as Taligent was still a going concern, it was difficult for new OS projects to gain any traction.

Instead, Apple continued adding new features to the same basic OS. During the early 1990s Apple released a series of major new packages to the system; among them were QuickDraw GX, Open Transport, OpenDoc, PowerTalk and many others. Most of these were larger than the original operating system. Problems with stability that had existed even with small patches grew along with the size and requirements of these packages, and by the mid-1990s the Mac had a reputation for instability and constant crashing.[17]

As the stability of the operating system collapsed, the ready answer was that Taligent would fix this - it was fully reentrant, truly multitasking, and made heavy use of protected memory. When the Taligent efforts collapsed, Apple was left with an aging OS and no designated solutions. Others were not suffering from the same problems; by 1994 the press buzz surrounding the upcoming release of Windows 95 started to grow to a crescendo, often with questions about Apple's response to the challenge it presented.[7] The press turned on the company, often introducing Apple's new projects as failures in the making.[18]

Another try

Given this pressure, the collapse of Taligent, and the growing problems with the existing operating system, with the release of System 7.5 in autumn 1994, Apple management decided that the decade-old operating system had run its course. A new system that did not have these problems was needed, and soon. Since so much of the existing system would be difficult to re-write, Apple developed a two-stage approach to the problem.

In the first stage, the existing system would be moved on top of a new kernel-based OS with built-in support for multitasking and protected memory. The existing libraries, like QuickDraw, would take too long to be re-written for the new system and would not be converted to be reentrant. Instead, a single pseudo-operating system, the "Blue Box", would keep applications and older code like QuickDraw in a single memory block so they would continue to run as they had in the past. The operating system itself would run in a separate partition, so crashing applications or extensions would not crash the entire machine.

Once the new kernel was in place and this basic upgrade was released, development would move on to re-writing the older libraries into new forms that could run directly over the new kernel.[19][20] At that point, applications would gain the modern features that had been available from Microsoft in Windows NT since 1993.

As System 7.5 was code-named "Mozart", the next-generation operating system that was intended to address the looming architectural issues was dubbed "Copland" after composer Aaron Copland. The follow-on system, "Gershwin", would complete the process of moving the entire system to the new platform.

Design

Copland was designed to run the Mac OS on top of a microkernel named Nukernel, which would handle basic tasks such as application startup and memory management, leaving all other tasks to a series of semi-special programs known as servers. For instance, networking and file services would not be provided by the kernel itself, but by servers which would be sent requests though interapplication communications.[21] Copland consisted of the combination of Nukernel, various servers, and a suite of application support libraries to provide implementations of the well-known Macintosh programming interface.[17]

Application services would be offered through a single program known officially as the "Cooperative Macintosh Toolbox environment",[17] but universally referred to as the blue box. The blue box encapsulated an existing System 7 operating system inside a single process and address space. Mac programs would run inside the blue box much as they did under System 7, as 'co-operative tasks' that used the non-re-entrant Toolbox calls. A worst-case scenario was that an application in the blue box would crash, taking down the entire box with it. This would not result in the system as a whole going down, however, and the blue box could be restarted.

Copland runtime architecture. The purple boxes show threads of control, while the heavy lines show different memory partitions. In the upper left is the blue box, running a number of System 7 applications (blue) and the toolbox code supporting them (green). Two “headless” applications are also running in their own spaces, providing file and web services. At the bottom are the OS servers, running in the same memory space as the kernel, indicating co-location.

New applications, those written with Copland in mind, would be able to directly communicate with the system servers and thereby gain many advantages in terms of performance and scalability. They could also communicate with the kernel to “spin off” separate applications or threads, which would run as separate processes in protected memory, as in most modern operating systems. However, these separate applications could not use non-re-entrant calls like QuickDraw, and thus could have no user interface. Apple suggested that larger programs could place their user interface in a normal Macintosh application, which would then start "worker threads" externally.[17]

Another key feature of Copland was that it would be completely PowerPC "native". System 7 had been ported to the PowerPC (PPC) with great success; large portions of the system ran in PPC code, including both high-level functionality, such as the majority of the user interface "toolbox" managers, and low-level functionality, such as interrupt management. However, there was enough 68k code left in the system, and especially user applications, that the operating system had to map some data between the two environments. In particular, every call into the Mac OS required a mapping between the 68k's interrupt system and the PPCs. Removing these mappings would greatly improve general system performance; at WWDC'96, engineers claimed that the performance of system calls would be as much as 50% faster.[22][23]

Copland was also based on the newly defined Common Hardware Reference Platform, or CHRP, which standardized the Mac hardware to the point where it could be built by different companies and run other operating systems (Solaris and AIX were two of many mentioned). This was a common theme at the time; many companies were forming groups to define standardized platforms to offer an alternative to the "Wintel" platform that was rapidly becoming dominant - examples included 88open, ACE and the AIM alliance.[22]

The challenge in Copland would be getting all of this functionality to fit into an ordinary Mac. System 7.5 already used up about 2.5 megabytes (MB) of RAM, and at the time this was a significant portion of the total RAM in most machines. Copland would be running what was essentially a complete copy of System 7.5 (in the blue box) and an entirely separate operating system running “under” it as well. Copland therefore was to use a Mach-inspired memory management system and rely extensively upon shared libraries,[24] with the goal being for Copland to be only some 50% larger than 7.5.

Development

The Copland project was first announced in March 1995.[25] Parts of Copland, most notably an early version of the new file system, were demonstrated at Apple's Worldwide Developers Conference in May 1995. Apple also promised that a beta release of Copland would be ready by the end of the year, for full release in early 1996.[25][26] Gershwin would follow the next year.[27] Throughout the year, Apple released a number of mock-ups to various magazines showing what the new system would look like, and commented continually that the company was fully committed to this project. By the end of the year, however, the developer release was nowhere in sight.[26]

As had happened in the past during the development of Blue/Pink, developers within Apple soon started abandoning their own projects in order to work on the new system. Middle management and project leaders fought back; they would claim that their project was vital to the success of the system, and move it into the Copland development stream—that way it could not be canceled and their employees removed to work on Copland.[28] This process took on momentum over the next year.

"Anytime they saw something sexy it had to go into the OS," said Jeffrey Tarter, publisher of the software industry newsletter Softletter. "There were little groups all over Apple doing fun things that had no earthly application to Apple's product line." What resulted was a vicious cycle: As the addition of features pushed back deadlines, Apple was compelled to promise still more functions to justify the costly delays. Moreover, this Sisyphean pattern persisted at a time when the company could scarcely afford to miss a step.[25]

In mid-1996, information leaked that Copland would have the ability to run applications written for other operating systems. including Windows NT. This feature had supposedly been in development for over 3 years. One user claimed to have been told about these plans by members of the Copland development team. Some analysts projected that this ability would increase Apple's penetration into the enterprise market, others said it was "Game over" and was only a sign of the Mac platform's irrelevancy.[29]

Soon the project looked less like a new operating system and more like a huge collection of new technologies; QuickDraw GX, SOM and OpenDoc became core components of the system, while completely unrelated technologies like a new file management dialog box (the "open dialog") and "themes" support appeared as well. The feature list grew much faster than the features could be completed, a classic case of creeping featuritis.[25]

An industry executive noted that "The game is to cut it down to the three or four most compelling features as opposed to having hundreds of nice-to-haves, I'm not sure that's happening."[30]

As the "package" grew, testing it became increasingly difficult and engineers were commenting as early as 1995 that Apple's announced 1996 release date was hopelessly optimistic: "There's no way in hell Copland ships next year. I just hope it ships in 1997."[30]

Developer release

At WWDC '96, Apple's new CEO, Gil Amelio, used the keynote to talk almost exclusively about Copland, now known as System 8. He repeatedly stated that it was the only focus of Apple engineering and that it would ship to developers at the end of summer with a full release planned for late fall. Very few, if any, demos of the running system were shown at the conference. Instead, various pieces of the technology and user interface that would go into the package (such as a new file management dialog) were demonstrated. Little of the core system's technology was demonstrated and the new file system that had been shown a year earlier was absent.

There was one way to actually use the new operating system, by signing up for time in the developer labs. This did not go well:

"There was a hands-on demo of the current state of OS 8. There were tantalizing glimpses of the goodies to come, but the overall experience was awful. It does not yet support text editing, so you couldn’t actually do anything except open and view documents (any dialog field that needed something typed into it was blank and dead). Also, it was incredibly fragile and crashed repeatedly, often corrupting system files on the disk in the process. The demo staff reformatted and rebuilt the hard disks at regular intervals. It was incredible that they even let us see the beast."[31]

After a number of people at the show complained about the microkernel's lack of sophistication, notably the lack of symmetric multiprocessing — a feature that would be exceedingly difficult to add to a system due to ship in a few months — Amelio came back on stage and announced that they would be adding that to the feature list.

In August 1996, "Developer Release 0" was sent to a small number of selected partners.[25] Far from demonstrating improved stability, it often crashed after doing nothing at all, and was completely unusable for development. In October, Apple moved the target delivery date to 'sometime', hinting that it might be 1997. One of the groups most surprised by the announcement was Apple's own hardware team, who had been waiting for Copland to allow the PowerPC to truly shine. Members of Apple's software QA team suggested, jokingly, that given current resources and the number of bugs in the system they could clear the program for shipping some time around 2030.

Cancellation

Later that summer, the situation was no better. Amelio complained that Copland was "just a collection of separate pieces, each being worked on by a different team ... that were expected to magically come together somehow."[32] Hoping to salvage the situation, Amelio hired Ellen Hancock away from National Semiconductor to take over engineering and get Copland development back on track.[33]

After a few months on the job, Hancock came to the conclusion that the situation was hopeless; given current development and engineering, she felt Copland would never ship. Instead, she suggested that the various user-facing technologies in Copland be rolled out in a series of staged releases, instead of a single big release. To address the aging infrastructure below these technologies, Amelio suggested looking outside the company for an entirely new operating system. Candidates considered were Sun's Solaris and Windows NT. Hancock reportedly was in favor of going with Solaris, while Amelio preferred Windows. Amelio even reportedly called Bill Gates to discuss the idea, and Gates promised to put Microsoft engineers to work porting QuickDraw to NT.[34] Apple officially canceled Copland in August 1996.[27] While the CD envelopes for the developer's release had been printed, the discs themselves had not been mastered.

After lengthy discussions with Be and rumors of a merger with Sun Microsystems, many were surprised at Apple's December 1996 announcement that they were purchasing NeXT and bringing Steve Jobs on in an advisory role.[35] Amelio quipped that they "choose Plan A instead of Plan Be."[36] The project to port OpenStep to the Macintosh platform was named Rhapsody and was to be the core of Apple's cross-platform operating system strategy including support for Power PC, Intel x86, and DEC Alpha CPU architectures, as well as an implementation of the OPENSTEP libraries running on Windows NT. This would in effect open the Windows application market to Macintosh developers as they could license the library from Apple for distribution with their product, or depend upon a preexisting installation.

Following Hancock's plan, development of System 7.5 continued, with a number of technologies originally slated for Copland being incorporated into the base OS. Apple embarked on a buying campaign, acquiring the rights to various third-party system enhancements and integrating them into the OS. The Extensions Manager, hierarchical Apple menu, collapsing windows, the menu bar clock, sticky notes — all were developed outside of Apple. Stability and performance were improved by Mac OS 7.6, which dropped the "System" moniker.[37] Eventually, many features developed for Copland, including the new Finder and support for themes (the default Platinum was the only theme included) were rolled into Mac OS 7.7, which was rebranded as Mac OS 8.

With the return of Jobs, this rebranding also allowed Apple to exploit a legal loophole to terminate third-party manufacturers' licenses to System 7 and effectively shut down the Macintosh clone market.[38] Later, Mac OS 8.1 finally added the new filesystem and Mac OS 8.6 updated the nanokernel to handle preemptive tasks. Its interface was Multiprocessing Services 2.x and later, but there was still no process separation and the system still used cooperative multitasking between processes. Even a process that was Multiprocessing Services-aware still had a portion that ran in the blue box, a task that also ran all single-threaded programs (and the only task that could run 68k code).

The Rhapsody project was cancelled after several developer preview releases, support for running on non-Macintosh platforms was dropped, and it was eventually released as Mac OS X Server 1.0. In 2001 this foundation was coupled to the Carbon library and Aqua user interface to form the modern Mac OS X product. Prior to the Intel release of Mac OS X 10.4 (Tiger), Mac OS X also used the 'rootless blue box' concept in the form of Classic to run applications written for older versions of Mac OS. A number of features originally seen in Copland demos, including its advanced Find command, built-in Internet browser, "piles" of folders and support for video-conferencing, have reappeared in recent releases of Mac OS X as Spotlight, Safari, Stacks, and iChat AV, respectively, although the implementation and user interface for each feature is completely different.

Hardware requirements

According to the documentation included in the developer release, Copland supported the following hardware configurations:[39]

See also

Notes

  1. ↑ Other sources put the date in 1987.

References

Citations

  1. ↑ Widman, Jake (October 9, 2008). "Lessons Learned: IT's Biggest Project Failures". PCWorld. Retrieved October 23, 2012.
  2. ↑ "30 years of the Apple Lisa and the Apple IIe".
  3. ↑ Adam Brooks Webber, the programmer responsible for porting TrueBASIC to the Amiga and Macintosh, Amiga vs. Macintosh, Byte Sept. 1986
  4. ↑ Engst, Adam C. (9 June 2003). "After Dark Returns for Mac OS X". Tidbits. Ithaca, New York. Retrieved 2013-09-11.
  5. ↑ Carlton, pg. 96
  6. ↑ Carlton, pg. 96-98
  7. 1 2 Singh, pg. 2
  8. ↑ Carlton, pg. 167
  9. ↑ Carlton, pg. 169
  10. ↑ Carlton, pg. 99
  11. ↑ Singh 2007, pg. 4
  12. ↑ "'Pink' may get a pink slip", Business Week, 1993, pg. 40
  13. ↑ Linzmayer 2004, pg. 69
  14. ↑ Linzmayer 2004, pg. 70 & 230
  15. ↑ Gordon Thygeson, "Apple T-shirts: a yearbook of history at Apple computer", Pomo Pub, 1997, pg. 44-48
  16. ↑ Linzmayer 2004, pg. 81
  17. 1 2 3 4 Dierks, 1995
  18. ↑ Quinlan, Tom (11 July 1994). "Apple set to ship System 7.5". InfoWorld: 6.
  19. ↑ Michael J. Miller (October 4, 1995). "Beyond Windows 95". PC Magazine. Retrieved 2006-07-23.
  20. ↑ Henry Bortman; Jeff Pittelkau (January 1997). "Plan Be". MacUser. Retrieved 2006-07-23.
  21. ↑ Francis 1996, pg. 32
  22. 1 2 Francis 1996, pg. 9
  23. ↑ Francis 1996, p. 18.
  24. ↑ Francis 1996, pp. 19, 20.
  25. 1 2 3 4 5 "Mac's new OS: Seven years in the making" cnet, 21 March 2001
  26. 1 2 Crabb 1995
  27. 1 2 "The Long and Winding Road", MacWorld, 1 September 2000
  28. ↑ Jake Widman, "Lessons Learned: IT's Biggest Project Failures", Computerworld, 9 Oct 2008
  29. ↑ "Computerworld Jul 29, 1996".
  30. 1 2 Burrows, 1995
  31. ↑ "WWDC'96: Looking for the Future", MacTECH, Volume 12 Issue 9 (August 1996)
  32. ↑ Gil Amelio and William Simon, "On the Firing Line", Harper, 1998
  33. ↑ Carlton, pg. 402
  34. ↑ "The Rise and Fall of Apple’s Gil Amelio".
  35. ↑ Dawn Kawamoto, Mike Yamamoto and Jeff Pelline, "Apple acquires Next, Jobs", cnet 20 December 1996
  36. ↑ Linzmayer 2004, pg. 277
  37. ↑ Singh 2007, pg. 6
  38. ↑ Beale, Steven (October 1997). "Mac OS 8 Ships with No License Deal". Macworld 14 (10). pp. 34–36.
  39. ↑ How to Install Mac OS 8 (D11E4), "Hardware Supported" section

Bibliography

External links

This article is issued from Wikipedia - version of the Wednesday, January 27, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.