Finally an SEO question on HN! While I haven’t worked on SO, I’ve worked on a medical equivalent site which started with 1500 of 2.5m pages indexed and when I was finished with my contract they had over 1.4m indexed.
There are a few factors that you need to adjust for: authority, quality, structure.
Authority is mostly out of your hands as a technical SEO scope but as more content gets indexed, more links and signals will come in. The real culprits are quality and structure. As others have said, low-quality pages need to be addressed. They can always be worked back in later but you’re better to be ruthless about it now and add them back later.
Profile pages, thin content pages, duplicates all need to go. No index them first, then block in robots eventually and doesn’t hurt to canonical them where applicable. If you jump straight to robots, they won’t pickup the noindex directive.
The structure, a lot of people think of folder structure as IA but really it’s linking structure. It’s important to know where you’re placing emphasis on pages by how you’re internal links are setup. This is also the best way to surface deep pages as you can link to them to bring Google deeper into your site.
Also you can try refreshing dates and seeing if that helps. Short term solution but works plenty well.
I know this is pretty general but it’s hard without seeing your issues specifically. If I can help you get on the right track, lmk. Gladly take a look. I run a small(free) community with some strong technical SEOs in it as well who like helping. Not sure how to connect but lmk if interested and we will figure it out.
Parents insurance provided us with https://getaloecare.com and it’s fantastic. Seems to be evolving but maintains it’s absolute ease of use for my parents who are luddites and 80+
There are a few factors that you need to adjust for: authority, quality, structure.
Authority is mostly out of your hands as a technical SEO scope but as more content gets indexed, more links and signals will come in. The real culprits are quality and structure. As others have said, low-quality pages need to be addressed. They can always be worked back in later but you’re better to be ruthless about it now and add them back later.
Profile pages, thin content pages, duplicates all need to go. No index them first, then block in robots eventually and doesn’t hurt to canonical them where applicable. If you jump straight to robots, they won’t pickup the noindex directive.
The structure, a lot of people think of folder structure as IA but really it’s linking structure. It’s important to know where you’re placing emphasis on pages by how you’re internal links are setup. This is also the best way to surface deep pages as you can link to them to bring Google deeper into your site.
Also you can try refreshing dates and seeing if that helps. Short term solution but works plenty well.
I know this is pretty general but it’s hard without seeing your issues specifically. If I can help you get on the right track, lmk. Gladly take a look. I run a small(free) community with some strong technical SEOs in it as well who like helping. Not sure how to connect but lmk if interested and we will figure it out.