Someone on a forum I go to mentioned how science fiction used to be popular in the 80s, but these days it's more about fantasy - Harry Potter and such.
This is a symptom of the change in human society, I think. Sci-fi as we know it began its great spurt in the 60s - there were SF authors before, like H.G. Wells or Jules Verne, but most of the classics of the genre as it is today, people like Asimov, Zelazny, Heinlein etc. started out in short-story magazines in the 60s.
SF fed on the public fascination with technology, the advent of the space age, and a general optimism about humankind's ability to control its environment and expand further. The correlation can also be seen in early SF - in Victorian times, which saw a great technological breakthrough and optimism - with the origins of modern fantasy (Tolkien, obviously) rising up after WWI, when humankind generally wasn't feeling very good about itself (cf). In the Cold War days, you had a clear demarkation of Good vs. Evil and Us vs. Them, which simplified things to a great degree. (People tend to think of the 70s as a time of disillusionment, what with the anti-war protests, violence in the civil rights struggle and such, but only a relatively minor percentage of the population actively engaged in that.) So the general mentality was optimistic: we'd gone to the moon, we were well on our way to Mars, and at least within the lifetime of our children we'd have a Brave New World of flying cars and such; interstellar travel would not be far behind. One can expect confrontations, but they are likely to be upon clearly demarkated moral lines - whether an empire of good vs. an empire of evil, or a rag-tag band of adventurers against a massing horde. Heroes included space generals, intergalactic spies, and individual frontiersmen who inevitably fought on the Good Guys' side, even if they did so begrudgingly.
As the Cold War came to an end, the Berlin wall fell and the Second World was in turmoil, Francis Fukuyama did that whole "end of history" dance; science fiction became darker, with people questioning their own actions. In the music world, Nirvana and its malcontent followers started a trend for misery; similarly sci-fi briefly floundered on soul-searching and dystopian themes (for example William H Keith's Warstrider series, or David Aikman's When the Almond Tree Blossoms), but it wasn't authentic.
So, with no faith in a glorious future, people started to look for a glorious past. A time of fable, King Arthur and elves and dragons and wizards. Worlds where the heroes were flawed, and victory unsure. If sci-fi characters battled evil and won through their own efforts, fantasy has deus ex machina; just when all seems lost, a hero comes from nowhere to rescue the day. People of the 90s and noughties have no faith in themselves, and want a benevolent wizard to come and rescue them.
Harry Potter is not a Tom Sawyer, making his way in the world with nothing but his spirit and wit; he is a regular, sad kid who suddenly got superpowers and now fights off shadowy demons with the help of his friends and teachers. This is the reality of the early 21st century, where most people's consideration of the world they live in is dominated by terrorism, religious fundamentalism and global warming. All I hope is that it is cyclical; that some time soon, we will get past this phase, and once again our hero will be the Stainless Steel Rat.