The following article contains my Extended Thoughts on "Bosses" discussed in the Gameology podcast with my co-host Mathew Falvai. You can listen to the Podcast via RSS, on iTunes, Google Play Music, or watch the episode in video format:
As much as we might hate tests in school, gamers relish the chance to rise to the challenge and tackle a well crafted boss. Although there are innumerable approaches to Boss design in games, there are some practices that have stood the test of time. Since different classes of Boss have differing best practices, I'll be dividing them into the following categories: Mini-Bosses, Temple Bosses, End Bosses, Experiential Bosses, and Secret Bosses.
Mini-Bosses are encountered by the player at mid-points in their experience. They might be guarding a special item, a door, or any other sort of objective. During the rising action of a given zone, they serve as a small climax, to help flesh out your ideal interest curve. Mini-Bosses can be anything from tougher versions of enemies the player has already encountered, to larger foes that fight in a unique arena that serves as part of their combat experience. Defeating a Mini-Boss in most Zelda games rewards the player with a new Item which they will learn to use before going up against the Temple Boss. Mini-Bosses can be constructed like other enemy characters in your game, and in fact they may be reused as enemies further down the game's timeline.
Other types of bosses usually should not recur in such fashion due to the nature of the challenges they present. The only place where it would make sense for these other kinds of bosses to re-appear might be in a "Boss Rush" experience, where the player must fight every single boss in the game one after another. Building a Boss Rush into a game's core experience can come across as padding out its length. Since bosses are usually unique enemies, it rarely makes canonical sense for the player to have to fight them again. Still, it can serve as a great gauntlet challenge for players to test their skill, so while I would advise against making a Boss Rush mandatory for progression, I would highly encourage their inclusion as a way for the player to earn some sort of rare item, or even as an unlockable bonus play mode that the player can access from the main menu of the game.
Temple Bosses serve as the big guardians of a given area in the game and perhaps a test of the player's mastery of a particular skill, usage of an item, or simply a "level gate" (evaluating that the player has reached an appropriate "level" before being able to proceed). By defeating this boss, the player is proving that they are ready for a new type or tier of challenge in the game. It is usually this tier of boss that is subject to the "3 hits" rule; not necessarily that the Boss dies in three strikes, but that the player successfully exploits the boss's moment of weakness on three separate occasions. Of course, this is specific to a game with a boss structure like Mario's; it wouldn't do to have a Dark Souls boss keel over in 3 hits. That being said, it is worth exploring why this is done, as many well-crafted bosses have been made with this sort of structure to them. To land the first hit, the player must familiarize themselves with the boss's attack pattern and moment of vulnerability. The second hit, on a smaller scale, is usually a smaller proof of mastery in itself, proving that the first hit wasn't a fluke. After the second hit, or once the player has brought the boss under 50% of its total health, it will enter an "Enraged" state. Once Enraged, the boss will often exhibit some extra challenging characteristics such as moving more quickly, attacking more aggressively, or changing up its attack pattern. Landing a final hit on an Enraged Boss ought to prove that the player has completely mastered the challenge the Boss present, and if the boss were to require further hits without continuing to change up its tactics, the fight could start to feel tedious. Since a Temple Boss fight is meant to serve as the climax of a particular zone, the last thing you want is for the encounter to feel like anything but an exciting challenge.
The concept of an "enraged" state goes beyond just Temple Bosses as well; End Bosses can have multiple stages to their encounters too. In the case of these multi-stage bosses, each sub-phase of the boss can (but doesn't necessarily have to) obey the rule of 3 Hits. If the battle is broken into stages like this, it can be a great opportunity to revaluate the player's mastery of all the skills they have learned throughout the game, with each phase of the fight testing a different skill. I personally believe these kinds of encounters are far more interesting than having a single large health bar which you must whittle down over time (an experience that is essentially static throughout the battle).
It's important to keep other types of "final challenges" in mind too. In place of a Temple Boss or even an End Boss, you can include a particularly challenging experience. A great example of this is the "warthog run" in the final levels of Halo 1 and Halo 3. While you aren't fighting any single large enemy, you are doing your best to survive a ship going into a nuclear meltdown and the collapsing superstructure of a Halo respectively. They provide a unique and memorable experience to close out the game and leave a (hopefully) positive impression on the player of the adrenaline fueled final moments.
All the bosses I've discussed so far are ones which fall somewhere along the planned path for the player. They should be designed to be within the skill range of the player at the moment they are encountered in the game, with perhaps the End Boss being a notable jump. Beyond that lie "Secret Bosses"; challenges that are far above the expected skill of the average player. Secret Bosses are typically found in RPGs, or any game where the player has free roam of the world. Even getting to these bosses is usually a gauntlet challenge in itself, like the Pit of 100 Trials in Paper Mario: The Thousand Year Door. The boss itself is typically more powerful than the End Boss of the game, and may require the player to have the most powerful weapons, armor, or abilities that can be unlocked in the game in order to succeed. If the End Boss tests a player's mastery of the "story" component of the game, the Secret Boss exists to test a complete mastery of the entire game, a true capstone challenge.
General Design Practices
When encountering a boss, especially a Temple Boss or the End Boss, you should consider the feasibility of introducing it with a cut-scene. This may not be necessary if the boss is a character which has already been introduced at an earlier point in the narrative, but even if that is the case, a brief cinematic will set the stage and heighten the player's anticipation for the battle ahead. You could use this cinematic as an illustration of the boss's power, or even make it functional by safely demonstrating the boss using one of its attacks while the player is invulnerable and giving some insight as to how the attack might be avoided. It also lets you show off the intricacies of the boss's design which the player might not notice if they are engaged in battle. Whatever the purpose of the cinematic, you should either keep it very short, or simply only play it the very first time the player encounters the boss, as it can be infuriating to sit through a cinematic multiple times if the player dies to the boss.
When looking to create a boss in a game of your own, it can be difficult to find the balance between something challenging enough that it will feel satisfying when defeated, and something which asks too much of the player. As with all aspects of game design, feedback and Play-testing are crucial to ensuring that players are understanding the challenge they are presented with. A boss is not the time to introduce new mechanics to the player, not when they could be within range of a one-hit-kill enemy and too distracted trying to survive in order to follow any new instructions. You must make sure the player can identify the boss's attack patterns and weaknesses (physical "weak points" or "moments" of vulnerability between attacks). However, when you go to implement that feedback, consider that instead of changing the boss, you also have the option of modifying the level leading up to it, helping the player to better understand the mechanic they are being tested on. You should want players to be able to defeat your bosses, as any good teacher wants their students to pass their tests.
Want your game design questions answered? Submit a question or comment to the Gameology podcast on BluishGreenProductions.com, and check out the Extended Thoughts articles while you're there. You can find me on Twitter @BluishGreenPro