It used to bother me that I was teaching literature to people who weren't in school to learn literature.
I decided to let that go as a problem in my life. Now what I think is this: I am giving some future physician, lawyer, or accountant one experience of thinking about literature that they wouldn't have had otherwise. I am it for that person, as far as literature is concerned (along with my other extremely meritorious colleagues, of course). If someone is interested in literature, then they can learn from me. If someone is moderately interested in it, open to it in some way, likewise. If they aren't interested, then they get to see someone who is.
One student who was into Spanish for the linguistics and teaching angle and wasn't too enthused wrote me later to apologize, because it took her until later, when she was in graduate school for linguistics, to understand what I was doing. But it was actually fine. I loved it that she wrote to me, but she didn't need to apologize.