Are end of level boss battles still needed in game design?

I still think you are being narrow in the definition of fun. No, it is not fun in the sense of "I feel happy that I am killing this innocent creature". It is fun in the sense of "This game is engaging me (emotionally, visually, mechanically, etc) and I have the desire to continue playing it". Maybe 'fun' isn't the perfect word for it, but that is what people are implying when they talk about a game being fun.

In the examples you give, the player is engaged by the game and wants to continue playing it, even if (s)he isn't filled with a warm and fuzzy feeling when doing so.

And in the context of this topic, boss battles that make the game fun are going to further engage the player, be it through providing gameplay pacing via a difficulty climax, or by acting as pacing mechanisms for the plot, or in some other manner.

 

I will be speaking from an RPG perspective, as it will be the easiest for me to connect the topic to.
Sorry that this is long. The summary is that boss encounters should be placed based on setting, plot, and level design. I explain below if you want to read more.

I think that the implementation and existence of a boss encounter is very dependent on the design of the level itself. Example: if you are a resistance fighter protecting your village from an attack, you will probably face (at the end of the level) a boss encounter in the form of A) a giant wave of enemies, B) the enemy captain, or C) the royal army's new prototype Giant Killer Robot Machine Weapon. However, if you're making your way through a nice, grassy hill, you probably won't run into anything too climactic (unless your script calls for it).

The setting and plot should be factored when considering if a boss encounter would be appropriate.

In considering setting, I like to think of enemies as being the "predators" of the area's ecosystem. A boss enemy, then, would be a creature at a very high part of the food chain, maybe even the top. For the purposes of creating tension, the boss's threat level to the player's party should be very high in comparison to the other inhabitants of the level.

However, you'll not always run into a tiger if you venture into the jungle. It could have been hunting somewhere else as you were bushwhacking your way through the foliage. If you encounter a fire-breathing dragon, it may be because you accidentally went into her nest. Maybe that giant golem you have to destroy is a guardian protecting the treasure you're about to loot. The thing is, a boss encounter in an RPG needs a reason to be there.

A third factor I would also consider is the fulfillment factor. Will the player feel rewarded for defeating X boss, or will X boss be a chore for the player? The extra boss may still be memorable to overcome, but a poorly placed (and designed) boss may take away from the experience.

To answer the original question, I would say that you shouldn't ask if you need a boss for a level; you should ask if a boss would make sense for the level. And to comment on the thread, I would replace the word "fun" with "memorable" when thinking about boss encounters. We can all argue what fun is to us, but there seems to be a generally accepted definition for the word memorable. A well-designed boss is memorable. It can be memorable for many different things: it was fun, or it looked cool, or it represented something very important to the game, or something else. Either way, there is a visible impact on the player's progression through the game.

Note that I am not a professional critic. I'm just speaking from my experiences with RPGs and adventure games. A lot of the things I spoke of do not necessarily apply to other game genres.

 

I like what @hippocoder has been saying, the question is not whether or not boss battles SHOULD be there, the question is whether they are fun or not. Games are not there to be symbols of intellectual reason, they're there to stimulate your emotions in a positive way and if they don't, they won't be played. Usually when I play games I'm looking for fast stimulation, I don't care whether they're built on top of some 'correct' principle. I think that's where a lot of indies go wrong, they think that just because something is original or rationally satisfying then it is going to be more important than e.g. sounding a nice, wet 'blip' when your character runs into a gem.

I think boss battles are important because they give a sense of urgency, focus and accomplishment to the game. I've seen too many ideas for games where its like "open world man, you can do whatever you like..." people don't want to do whatever they like (except to some extent), they want to be tested, stimulated and rewarded. Bosses are a great way to do that.