Writing and (hopefully) selling short stories has really made me think about why people read. As unrealistic as it seems, every author, I think, secretly hopes to create that ONE book that is universally loved and immortalized by literary history. Most of us are practical enough to realize that will never happen. Even JK Rowling has haters.
Subject material is definitely something that affects how readers will perceive your work. If a reader doesn't like fantasy, there's not much you can do about that. But a lesser thought of concept is why people read. This actually varies almost as much as subject material.
Some people read almost exclusively to inform themselves. They love non-fiction. The idea of reading about dragons or lasers seems like a silly waste of time. Some people only read entertain themselves. They want a really shallow book that doesn't take very much thought juice to figure out.
This are extreme examples and most of us tend to vary in what we feel like reading at different times. But I think that this is really important to keep in mind when writing. Just because you write that fantasy doesn't mean that all fantasy readers will love it. Some fantasy readers love the heavily detailed worlds that require the reader to make charts in order to figure out. Other fantasy readers may just want a more straightforward good guy vs. bad guy story.
All aspects of your potential reader should be considered. Don't just market to a fantasy audience only. Think about the other elements too. A reader that's in the market for light, fun fiction might be willing to experiment with a different genre so long as your book provides that type of entertainment.