Who Decides What's Interesting?

AI researchers are working hard to teach machines what's worth finding interesting. But when you use a language model as a stand-in for human curiosity, whose curiosity are you actually capturing?