No, not a death match between the two genres, nor even a discussion of which is “better” in some way. I like both in different ways. Each suits a mood for me. No, this is rather about when something is one or the other. This will be something of a ramble.
Some folk have given long, involved definitions about when something is Science Fiction and when it’s Fantasy. Me? I like one similar to Orson Scott Card’s from one of his writing books. Science Fiction has rivets and engineers. Fantasy has trees and elves.
The late Arthur C. Clarke in his “three laws of prognostication” gave as his third law that “Any sufficiently advanced technology is indistinguishable from magic.” Some folk, have inverted that: “Sufficiently advanced magic is indistinguishable from technology.” Those twin statements are actually quite telling in looking at the fuzzy border between fantasy and science fiction.
A lot of it has to do with mindset, both the mindset of the writing and the mindset I fall into while reading it. Sometimes a book can be both or either depending on how you look at it.
Take, for instance, the late Anne McCaffery’s Pern books. They are science fiction. A colony ship reaches Pern, an almost idyllic planet. However, once the colonists have settled in and are essentially committed, an unexpected problem arises. Another planet in the system, one with a highly elliptical orbit nears the sun and, for reasons that are mostly glossed over, extremely aggressive fungal spores cross the gap between this other planet and Pern. The spores, called “thread” cause serious destruction, basically “eating” anything organic they hit, but are fortunately short lived so that they don’t completely lay waste to the planet. Still, this is a disaster of epic proportions for the colonists. A biologist on the planet engages in an emergency program of genetic manipulation, taking an indigenous species of flying lizard that has already demonstrated the ability to imprint on people at birth (forming an empathic bond) and not only augmenting that imprinting ability to a true telepathic as well as empathic bond and increasing their size, forming human carrying, self-replicating flamethrowers–dragons.
This is far backstory, however, for the first published Pern stories. When we’re introduced to them, the world and its characters, due to a number of crises over the years, are essentially in a dark age and have forgotten much of their history and science. So it’s a pre-industrial age with dragons and dragonriders.
Truth to tell, even knowing the back story, even having read the key prequel that told the story of landing and the first dragons, it still reads like fantasy to me. My “mindset” while reading it is the one that I use when reading other fantasy. The “fantasy elements”–the telepathic bonds, the ability of the dragons to go “between” (teleporting) are decoupled from the in-story “science” and they become the functional equivalent of magic.
On the flip side you have Rick Cook’s “Wizardry” books. Here, Rick Cook has a clearly magical world but the main character, brought in from an analog of the “real world” takes a scientific approach to that magic, treating it like computer programming where small spells are created that function as functions, routines, and lines of code. By bringing a scientific approach to the magic, it in many ways reads more as science fiction.
Similarly there is the late Poul Anderson’s Three Hearts and Three Lions. The main character is once again taken to a fantasy world and approaches the magic of the world in an analytical way that unveils the deep thought Anderson clearly gave the magic of that world. As one example, when the protagonist tricks a Troll into staying out past sunup and it is turned to stone, he realizes why Troll Gold is considered cursed. The transmutation of carbon into silicon (the conversion from living flesh to stone) leaves the gold highly radioactive. Anybody carrying it would soon sicken and die.
And so, this, too reads more like Science Fiction in many ways.
Now, consider Star Trek and Star Wars. From the standpoint of modern physics, they are both ridiculous. No, “reverse the tachyon flow” is no more scientific than “use the Force, Luke”. (Someone basically just threw out the idea of “tachyons” from looking at the relativity equations. If some particle had an imaginary rest mass and were traveling faster than light, in relativity that would give it a real momentum and a real energy. There’s no evidence that tachyons exist. And there’s nothing in physical theory that says they must, or even should, exist. They’re just an idea someone tossed out in pure speculation.)
The two series’ have a lot in common. Space travel. Alien worlds. Faster than light travel. War, sometimes. Exploration, sometimes.
However, there’s a big difference between the two series. In Star Trek the presumption is that the fantastic elements are the result of science and engineering. Research will (in the story world) lead us to those discoveries. Scientists will find them. Engineers will build them. In Star Wars there is a lot of stuff that is built by science and engineering, but the story doesn’t center around that. It centers instead around mysticism and, frankly, magic. “The Force is what gives a Jedi his power.” “Your sad devotion to that ancient religion…” “‘You mean it controls your actions?’ ‘Partially, but it also obeys your commands.’” And as the franchise developed, these mystics, these “space wizards” central even from the beginning of the series (from “Help me, Obi Wan Kenobi, you’re my only hope” to “You’ve switched off your targeting computer, what’s wrong?”–this mystical Force, this space magic and it’s users, were the key MacGuffin) grow to dominate. It’s not the engineering and the science behind it that is central to the Star Wars universe as it is in Star Trek. It’s the space magic. Even the light sabers, a cool piece of technology in the beginning of the franchise, are quickly revealed (in the Expanded Universe) that one needs to use the Force to properly align the crystals at the heart of their operation. They’re not cool tech any more. They’re magic swords, forged by wizards.
So while both franchises have the trappings of science fiction, Star Wars, in many ways, has more of a fantasy feel. But those trappings are enough for many people to still see it as science fiction.
And, so, in the end, it really comes down to the eye of the beholder.