I actually think HN achieves the illusion of greatness via excessive moderation. It’s very easy for users to flag a topic from the front page and (IIRC) there are even automated downranks for threads that attract a lot of back and forth arguing.
The result is a site with relatively measured debate but also a large chunk of missing debate. All that said I don’t have some genius idea about how to do it better so my criticisms can only go so far.
Sure, some debate is missing here. But I think it's fair to posit that the debate in question is of low value, and that we're better off without it. Or, at worst, that losing it is a wash in the grand scheme of things.
In either case, the owners/operators of HN set a standard for the kind of discourse they want to encourage, and they take steps to encourage it. Nothing wrong with that. Reddit, Slashdot, 4chan, Twitter, and countless other sites exist for people with different tastes.
I think that in general the plain linear format is bad for debate. You always end up with one or two - often barely on topic! - threads dominating, and everything else below gets barely any attention.
For submissions with lots of comments the majority of comments are all under the top comment, branching off into all kinds of places.
It only works when there is not a lot of participation, unless you count participation itself as the goal.
There is also the limit of usefulness of longer comments, a few paragraphs - not too little, but also not too much - and everything unstructured severely limits the quality of the information.
You have many layers but it is all pressed into the linear format. You usually have less than a handful of actually on topic posts that really add significantly to the OP submission, but you may also have a lot of great discussion that is adjacent. Now, it is hard to find those few great on-topic posts, and people who may have something interesting to say may not even do so when they see they are comment #200+ because they know they won't be read far down the list.
There is also no connection in time. Every single comment only gets a brief moment in space (the submission's comment space) and time (a few hours at most before nobody will ever see it again). You can't build something up over time, it's all quite superficial.
That is not HN specific, all comments sections are the same everywhere. I am disappointed by the lack of innovation in this space. It reminds me of how many everyday things are not improving, for example, bad public toilets. I see the same ones clogged again and again, and dirty and stinking. And yet, they never make any changes, and the new constructions all have the same problems. For example, why does cleaning them remain such a disgusting chore? They could just have better surfaces, a hose and a drain so that the cleaner or even a user can just use the hose to clean a stall or the entire room. I saw that in a sausage factory when I worked there while in school, hot water hoses you could use any- and everywhere, and indeed everything was very clean because everybody just used a hot water hose a lot.
I no longer believe in automatic improvements, and it has nothing to do with whatever system society uses. There is just a lot of inertia, things just continue and nobody really spends much effort to improve many easily improvable things. Part of it is not just starting something, it's also the (correct) expectation that even if you provide something better it will be shutdown, and it has nothing to do with price (cleaning and unclogging those toilets must add up over time, never mind that e.g. the big 10 theatre cinema owner should have the idea that maybe customers always having to go into stinking restrooms is bad for business long-term?).
Back to comment systems, we have TONS of comment systems, but they all do pretty much the same. I don't believe for a second that there is no other way and we have a global maximum. We have a local maximum, and we cannot seem to get out of it.
The result is a site with relatively measured debate but also a large chunk of missing debate. All that said I don’t have some genius idea about how to do it better so my criticisms can only go so far.