It's not your imagination. The reason you have no interest in reading
the vast majority of books on the shelf is because they are not written with you in mind, published with you in mind, marketed with you in mind, or sold with you in mind. And yet, the publishing industry affects to be surprised that you do not buy their products.
This NPR piece three years ago came to the conclusion that women read more fiction than men by a 4-1 margin. Articles like this madden me because I think they miss the big picture, or perhaps are even ignoring it purposefully. It's like discussing global warming, while completely ignoring the fact that hey, maybe we have something to do with it.
Nobody can deny the fact that most editorial meetings tend to be dominated by women. Saying the ratio is 75/25 is not overstating things. So needless to say when a male editor pitches a book aimed at men, there are perilously few men to read it and give their opinions. Not to mention that, because there are so few men, the competition to buy books aimed at men is astronomical. I was once shot down in an effort to buy a sports humor book because I couldn't get the support of a senior editor. The reason? This editor had written a similar book proposal on submission and didn't want to hurt his chances of selling it.
Men read. Tons of them do. But they are not marketed to, not targeted, and often totally dismissed.
There is, of course, one other factor that may prove to be at least somewhat significant now that ebooks are so popular. Men tend to know how to pirate things. I buy a fair amount of books because I like being surrounded by real books on bookshelves. But I don't actually need to buy most of those books, I only choose to buy them in order to support the author and encourage the publisher.