[In this, the final installment of Where Games Go To Sleep, Microsoft, Nintendo, Sony, and other leading video game companies respond to a video game preservation questionnaire, disclosing how they're saving their video game history. Developers and publishers disclose what assets they've lost, and how, in certain instances, have rescued and preserved the building blocks of their games before it was too late.]
Any video game enthusiast can browse the internet or track down a print magazine or video game book to read quips or unconfirmed rumors about how the game industry has lost the very material it created. Parts one and two of Where Games Go To Sleep: The Game Preservation Crisis provided just a few of these anecdotes from its interview subjects, ranging from Atari Corporation source code being given away in an office equipment sale, to Sega source code being lost, but then found in an ex-staffer's garage.
The decision was made to reach out to the video game industry directly and formally ask: does video game preservation matter? As a developer or publisher, how is your organization preserving its video games for the future?
A special questionnaire made up of four questions was sent to developers and publishers worldwide. The developers and publishers that were sent a questionnaire were also given the option to submit one whole statement that sums up their stance on game preservation in lieu of answering the four questions.
Certain developers and publisher were asked extra questions related to their titles being rereleased on retail compilations and via gaming subscription services.
61 video game developers and publishers worldwide were contacted via postal mail beginning in 2009, and then contacted once again via email throughout 2009-2010. A total of 14 video game developers and publishers responded. In the final part of this article these 14 video game companies reveal just how they're preserving the games they've developed and published, games that have provided immense entertainment over the past three decades.
The final part of this article reveals their answers and statements in a condensed presentation, while their complete answers and statements can be read in their entirety here.
Some video game industry personnel were very enthusiastic to participate, while some declined to participate entirely, citing company policy in disclosing development matters. It's presumed that in some cases, our questionnaire may have ended up being lost in the mail, trashed in the spam folder, or set aside altogether due to time constraints.
As mentioned in part one of this article, this was a very difficult subject for the industry to openly discuss. This became apparent when final answers and statements were submitted for this article, some of which revealed startling realities of what's already been lost in the video game industry. Fascinating stories of what's been saved were also shared.
It was an absolute honor to hear from the companies that responded -- their time and effort providing answers or statements was very much appreciated. For those in the video game industry that missed on our initial game preservation questionnaire, they are invited to send their answers or statements to the author directly. Instructions to submit answers or provide a statement are provided at the end of this article, and any answers or statements received may be used in a potential Gamasutra industry-focused follow-up to this feature.
Console manufacturers Microsoft, Nintendo, and Sony were gracious enough to share their views on the subject of game preservation. Capcom, Digital Leisure, Gearbox Software, Intellivision Productions, Irem Software Engineering, Jaleco, Mitchell Corporation, Namco Bandai Games, Sega Corporation, Taito, and Throwback Entertainment (majority owners of the Acclaim Entertainment library) were the video game developers that also submitted answers or statements providing their stance on how they're preserving their arcade and console titles.
An original "Puck Man" (aka Pac-Man) cabinet sits on display in the Tokyo office lobby of Namco Bandai Games. The author wishes to thank Izumi Wada of the Namco Bandai Games corporate communications department for his invaluable assistance and time in capturing these images.
Ken Lobb of Microsoft Game Studios revealed how the company utilizes special departments to store all game software and hardware. Multiple copies of each game published by Microsoft (making up source code and production materials) are stored in humidity- and temperature-controlled environments in both onsite and offsite locations. Microsoft has plans to transfer games published prior to the year 2000, (stored on older media) to a more reliable storage solution.
"The source code and all the materials used to build the products for the games released after 2000 are already stored on highly reliable present-day storage media, in secure, temperature & humidity controlled locations. Retrieving the source code and even rebuilding the games is a part of our comprehensive business continuity and disaster recovery (BCDR) program," says Lobb.
Many of the tools used to preserve Microsoft Game Studio source code were developed within the company. Microsoft is open to provide "general guidance" on its best practices for source code preservation to video game-related industry organizations, while making it clear that it would not be able to share "specific encryption/decryption algorithms".
Marc Franklin, public relations director of Nintendo, highlights in a statement how its legacy has played an important part in its present-day game releases: "Our games reach back decades and star dozens of characters who are still going strong today. Plus some of these older games introduced genres, styles, and technological breakthroughs that are now commonplace.
"As we highlight in our Iwata Asks [interview] series, Nintendo keeps a wealth of materials related to its past games, up to and including even original design sketches and documents. Preserving these games lets us reintroduce them to new players while giving older gamers a chance to relive their glory days."
Sony Computer Entertainment of America disclosed how both its IT and QA groups each play a role in archiving game source code and assets developed internally, continually transferring data to current storage media. Sony's storage management and methods vary per region. Archiving externally developed game titles also varies depending on contracts.
Sony Computer Entertainment did outline three specific challenges of archiving completed products, challenges it will face as hardware and production software changes stating:
- We need to find a way to archive PC hardware and software products that have hardware keys. BIOS expiry can cause problems with long-term storage of PC hardware and by inference development tools. We need to assess how we can maintain or reflash BIOS revisions for older chipsets. EPROMs often expire within 10 years or so.
- Legacy music and video source materials may be in file versions that require specific hardware (cards or other peripherals) which are no longer available. We need to identify how media companies that manage archives deal with this.
- Devkits and hardware tools need to be stored alongside game data in a secure disaster-proof physical location.
Developers and publishers from Japan also affirmed the importance of storing their game data and production materials in disaster proof locations. Jaleco highlighted fundamental points of protecting their game data in locations that are safe from fire, natural disasters, dangerous climates and other security threats. Jaleco, as well as Taito, ensure that its operations meet ISO standards. (Jaleco specifically mentioned they meet the ISO27001 standard.)
Taito stated that preserving its video games is "vitally important", and that it follows a "protection" and "foster" policy. By "protection" it intends to guard its games against copyright infringement, and by "foster" the company wishes to build up what it calls a "sound video game culture". Taito views preservation and "protection" as the same, which encompasses rereleasing its games onto console compilations and mobile platforms as well as other non-traditional platforms and media.
Taito disclosed that no preservation policy ever existed in the "early video game age": "We have problems for the recouping the game data and their proper preservation."
Taito does, however, maintain that for its arcade games in particular, it keeps the hardware itself together with the ROM and printed circuit boards.
Taito also admits that even though it utilizes diagrams and other engineering materials that are needed for extended user/customer support, many of their video game promotional materials have been lost over time.
Just as Taito disclosed that no such preservation policies existed early in its company history, Former Capcom producer Ben Judd emphasized this very point, as he learned that many Japanese video game publishers did not institute storage and preservation procedures until the early 1990s:
"Capcom does have a procedure for preserving their code. However, these procedures, like many other Japanese publishers, were not instituted until around the 16 bit days. I have actually seen massive stacks of dot-matrix printer paper that contained game code on them.
"On a side note, I've heard of incidents of old game ROMs from other publishers that have disappeared and are forever lost. I was quite surprised to initially hear this, but I was even more shocked when I found out that it is quite common for code for a lot of big name 8-bit titles from other publishers to be gone," says Judd.
Irem is one Japanese developer that openly admits they have no intact source code from games developed and published in the 1980s. Irem expressed concerns that even the game ROMs themselves are in danger:
"Although we keep almost all ROMs from 1980s, some of the hardware for these ROMs is broken down. Since the hardware has been drastically changed, it is very hard to repair the hardware (not only are the parts missing but also the repair engineer is missing)," comments Kenta Sakai of Irem Software Engineering.
A JAMMA PCB held in storage at the Tokyo offices of Mitchell Corporation of the 1991 arcade game Drift Out. Originally developed by Visco, Drift Out was formerly distributed by Visco sales agent Mitchell Corporation.
Namco Bandai Games, Sega, Mitchell Corporation (along with Capcom and Irem) were among the other Japanese video game developers that also responded to this questionnaire. Each company shared the belief that preservation of its games is important, and stated that they have transferred many of their older game titles to present-day storage media, and store them in safe, secure, climate-controlled environments.
Digital Leisure, Intellivision, Throwback, and Gearbox Software were among the North American developers that discussed the challenges of maintaining their games while dealing with numerous technical challenges.
Digital Leisure, in handling the laserdisc format of games including Dragon's Lair and Mad Dog McCree had to intricately put their games back together using a number of different sources, as Digitial Leisure's supervising producer Paul Gold explains:
"Much of the original code source code for our classic IP like Dragon's Lair and Mad Dog McCree was either lost or not accessible due to obsolescence of the media it was stored on. We worked with a variety of people and fans of the series to properly piece back together how the original games played to create arcade authentic versions of the games.
"However, the source code for these titles isn't as relevant, as the actual video which no matter the format can be transferred in a digital world with ease. Dirk the Daring doesn't need a normal map or a vertex shader to thrill audiences like he did in the '80s."
Not only was Digital Leisure dealing with source code, it also had to produce an HD re-master of the Dragon's Lair animation using original negatives retrieved from film vaults for new rereleases of the game. The efforts that Digital Leisure put forth in the remastering and preservation process have ensured that future developers and publisher could reuse the material:
"When we decided that the Dragon's Lair series would be a great title to remaster for HD formats, we contacted Don Bluth Films who, like most studios, had been storing the original negative in a film vault for over 20 years. The negative hadn't been handled since 1983 and was in such a delicate state that we decided it would be best to create an all new inter-positive from the master, so should someone wish to release it in the latest format of the day 20 years from now -- they can.
"It was also incredibly important that the color for each of the games was properly corrected so there were no inconsistencies. In fact upon release of the HD version, fans could not believe how rich the colors really were, as the video looked better than what was originally on the arcade machine!"
Gold also discloses that Digital Leisure has been unable to obtain the rights to rerelease other laserdisc games due both to the fact the original source material no longer exists, and ownership rights of other material are unknown.
Keith Robinson, president and co-founder of Intellivision Productions recalls how he had to go into rescue mode when retrieving the source code of Intellivision games, having to track down old hardware that former corporate owner Mattel had sold at auction.
"When Mattel shut down game production in January 1984, two of the programmers, Mike Minkoff and Mike Breen, were kept on for a month or two to archive all of the finished games and games in progress. This wasn't for posterity; Mattel hoped to find a buyer for Intellivision. The source code was backed up onto 8-inch floppy disks.
"When INTV Corp purchased the rights, it got the disks. INTV contracted another of the programmers, Dave Warhol, to complete some of the unfinished games, so he needed the source code transferred from the 8-inch floppies to PC-readable 5 ¼-inch floppies. He hired me to do that job.
"The problem was that the Mattel Electronics disk drives used a non-standard discontinued format. A year after Mattel Electronics closed, no one could read the disks. I contacted Mattel Inc. and tracked down the original drives: they had been sold at auction to a company in the northern part of the San Fernando Valley.
"It turns out that they had replaced the controller cards in the drives to make them compatible with other 8-inch drives. But they still had the old circuit boards in their attic and were willing to put one back into one of the drives. It didn't work until I called the manufacturer and got the correct configuration for the jumpers on the board.
"So we successfully saved the Intellivision source code. When Intellivision Productions bought the rights in 1997, we got the 5 1/4-inch disks from Dave. We then transferred them to 3 1/2-inch PC disks and later to CDs."
Robert Maduri, CEO of Throwback Entertainment, faced the same technical challenges when Throwback acquired a vast majority of game titles from bankrupted Acclaim Entertainment. Countless Acclaim assets that accumulated at the company's former Glen Cove, New York office had to be sorted and organized.
"Upon the closure of the Acclaim acquisition, Throwback faced a pretty daunting task of assembling data that spanned over 280 product SKUs, dozens of hardware platforms and mass quantities of data stored on obsolete storage devices. For a library that generated over $3 billion in revenue, every piece of data was critical.
"Quite honestly, it was a logistical nightmare. We found that as long as sufficient backups were created, not a lot of thought was put in to the progression of technology and updating the files as the native environments changed. Prior to this past generation of consoles, what was the point of doing so for the bulk of the industry?
"When we acquired the library, it was eerily similar to acquiring a traditional library. We had differing native file formats, disk mediums, and file systems that required tending to. It literally spanned over 25 years of computer technology. After trying various applications and hardware systems with the intent to import the files, we quickly learned that the best way to tackle the problem was to create a data center that was also an accumulation of the past 25 years.
"eBay was a life saver in this regard. Everything that would be considered obsolete quickly proved its value ten times over. Computers, networking systems, development hardware and external drives being shipped from around the world became the norm. What was old quickly became new again.
"Once we were able to view, catalogue, and manipulate the data in their native environment, the rest of the process became that much easier."
Michael Athey, chief information officer of Gearbox Software, notes that all Gearbox employees contribute to day-to-day preservation efforts in saving everything in its corporate culture, from marketing and product support materials, to preliminary game designs and promotional print items. When it comes to actual game data preservation, Athey reveals how Gearbox utilizes multiple locations to accomplish this feat:
"Most everything is stored digitally on our local SAN and replicated to sites both internal and external to Gearbox. wWe even have multiple locations storing physical SKU items of our titles. All of this is done for both nostalgia purposes and business continuity.
"As of now, all of Gearbox's data is on redundant D2D solutions, both on and offsite utilizing D2D de-duplication technology, so that we don't have a heavy investment in offsite storage. We had a bit of a headache a few years ago getting our older titles, source, and support assets off legacy tape media that they had been stored on for years prior.
"We took the time to pull the data off, verify integrity, and then move them permanently to our current storage technology. We could have left the data on the tapes but felt it better to have a consistent plan for storage and retrieval of all data."
Many video game developers agreed that the industry should work together in researching preservation methods through events such as GDC and sharing information via game-related industry organizations. Meanwhile, companies are concerned about the expiring lifecycles of development software and hardware that poses a constant threat to preservation.
A few companies did reveal that copyright and trademark issues (locating game copyright owners and resolving ownership disputes) are among other obstacles that also need to be overcome. Some publishers did feel that the responsibility of video game preservation ultimately lies with each individual developer and publisher.
Michael Athey of Gearbox Software sums up a viewpoint that many questionnaire respondents expressed when asked if the industry as a whole should work towards researching game preservation solutions:
"It would be in the best interest of the industry and its service to the customers and fans if we worked together to at least have a framework on how to archive video game titles, both its raw assets and code, and final product. This is good for longevity of the titles, rebranding of intellectual property, and possible continuation of ROI through third party exploitation. Who knows what title from the past may show up on some piece of fancy new technology in the future?"
A Japanese coin-op cabinet for Xevious, designed by Masanobu Endo and one of the first original scrolling shooters, sits on display in the Tokyo office lobby of Namco Bandai Games.
Your Response is Requested
Video game industry developers, publishers and personnel that were initially unable to answer these questions are invited to answer these questions for a potential follow-up article to Where Games Go To Sleep.
The four questions posed to video game developers and publishers in the original questionnaire were:
1. Is it important for _____________ to preserve its video games for future audiences?
2. Movie studios preserve their film negatives by storing them in special vaults to protect them from natural disasters and other harmful elements such as humidity.
Does _____________ preserve their source code, hardware, and production materials* in special vaults or locations to protect them from natural disasters and harmful elements?
*These production materials can include: Instruction booklets, marketing assets (print advertisements, TV commercials), and any or all production/development materials (character designs, dialogue scripts, and level layout designs).
3. With regards to video games, the greatest threat to video game source code is data/bit loss, and overall hardware failure and/or obsolescence.
Could _____________ explain any specific challenges of maintaining, transferring or retrieving video game source code from the 1980's or early 1990's hardware/media? (An example is transferring data from floppy magnetic disks that are vulnerable to damage over to new and more reliable present-day storage media).
4. Should the video game industry and its related industry organizations work together to research solutions on how to properly preserve, archive and store video game source code and production materials?
Please forward your answers to these four questions via email with your name and contact information (for confirmation purposes) to the following email address: [email protected].
Any developer may choose to answer each question individually (which is preferred), or submit one whole statement that addresses its overall position on video game preservation.
The author of Where Games Go To Sleep would like to sincerely thank all of our video game industry participants and archivists who took the time to answer questions for this article.
You can download the full survey results for all publishers included in this article as a Word document here.