Even if memes are not genetically heritable, they are still transmissible, and therefore still subject to selection pressures. And the more children these families have, the broader the attack surface for large-family beliefs to infect the next generation.

Moreover, I would suggest that genetics do influence tendencies for adopting some memes over others, especially as regarding the instinct to spread genetic material, which is very ancient. I can't cite any scientific support for this idea, but the likelihood seems very strong.