I know I did a blog entry before on believability, but I’m going to focus more on Fantasy with this one, since it’s something that’s come to my attention that many people that write fantasy have issues with this.
I’ve heard on a number of occasions the argument that “This is fantasy! I can do whatever I want!” This is, to an extent, true. But the fact is that if what your character is doing isn’t in the slightest bit believable or “real” to the reader? No one will be interested in how “awesome” it is.
The fact of the matter is that while your novel may be “sci-fi” or “fantasy” or whatever else you might want to call the genre you’re writing in, you have to make the world believable. For example: physics. Whether the physics of the world you’re writing in are the same as those of the one we live in or not, they must have rules. And you must tell us what those rules are if they are not the same rules that earth operates under (like a planet with more, or less, gravity than earth, etc.).
This same rule regards systems of magic (or mental powers of any kind) or whatever possible system you operate under. The Jedi certainly have rules regarding their abilities, as do the wizards in Lord of the Rings. These rules, while you may not have to spell them out word for word to the reader, must be adhered to unless breaking them is an intrinsic part of the plot (if it is, go for it!).
Believability also comes in the form of making sure injuries are handled realistically and the same with character reactions. If your character has never killed before and they just slew someone, is it natural human reaction to simply shrug and walk away? Not usually. Now, if your character is a Vulcan (nerd-powers activate), then it might be. But if they are, that’s believable for that race. So your characters certainly don’t have to behave in a way that’s human if they’re not. In fact, it may be more believable for them to react in ways that aren’t. So if you have an elf, have him react in the way that the elf race you are writing about would.
If you’re going to have races that aren’t human, make them believable by giving them depth, history, and culture. If you’re having trouble with that, there’s a book by Orson Scott Card on how to construct societies and such for fantasy writers that is absolutely worth looking into. It’s called “The Writer’s Digest Guide To Science Fiction and Fantasy” and it contains a veritable wealth of information that any writer of these genres will find immensely useful. It’s available on Amazon:
http://www.amazon.com/How-Write-Science-Fiction-Fantasy/dp/158297103X
Believability also needs to come from the surroundings. If you’re going to have a “high fantasy” novel, you need to make sure that your understanding of the real world time period (usually Medieval or Renaissance, for “High Fantasy”) is solid so that you don’t end up making classic blunders unless you’re specifically relying on those tropes in order to fit the genre. I’d suggest doing research on that time period (Card actually has a lot of useful information about that in the book I mentioned earlier) and what they ate, drank, and did with their time. Making assumptions or thinking you know about it because you’ve watched enough episodes of “The New Adventures of Robin Hood” or “Xena: Warrior Princess” will do nothing for you. That kind of thing rarely portrays accurate historical reference for anything, even if it’s fun to watch (yeah, I watched Robin Hood obsessively as a young teen…).
One of the best examples of a believable world (in my opinion) is the series of books set in Krondor by Raymond E. Feist. He’s one of my favorite authors largely because of this. The series is high fantasy, but the world it’s set in has such rich depth and realism that you find yourself almost believing it. Personally that, to me, is the mark of a very skilled writer and should be your barometer.
Thirteen Cents Short 
Another good example is Song of Ice and Fire. It’s not in the real world, but the characters and history are so fleshed out that you can believe it.
I’ve actually gone and done an upheaval of my own work based on having “the rules” be broken as part of the plot. That’s also a way to make it so your character isn’t just some “Rolf the Nobody” who happens to blunder into an adventure for no reason.
I’ve heard of that series (?) but never read it, though I would be willing to bet that it is good! I’ll have to add that onto my “to-read” list. Which is going to take me until I die, probably.
Breaking the rules is always fun, so long as it’s done with reason. Half the best plots in the world are people that broke the rules, but there was reason for it and that’s what made it believable and a good read. 🙂
Yeah, my own reading backlog is probably exceeding the maximum human life expectancy by this point, but I don’t mind now that all the books are the ones I want to read. Like Discworld, which is another good example of pretty much exactly this.