Matt's a known liar, and that 4000/hours a week includes a lot of stuff that doesn't involve working on code. (For example, Matt is including time spent censoring the WP forums, and harassing event sponsors in that time.) Given that the bulk of the WordPress team quit last year, I'd be surprised if they're even spending a 100 hours/week as a company right now on anything related to code right now.
But if we're going by hours...the WP Community as a whole probably spent several hundred thousand hours on maintaining or improving WP last week.
The staff that quit was the staff that worked on WP. Almost none of the remaining employees worked on it, so yeah I would not be surprised if only 4 people are working on it now and that's the real cause of Matt pulling back on their WP work.
Automattic previously spent ~4k hours a week maintaining wordpress, so I'm not sure what you're on about