I think the point is that the vast majority of people don't really have a unique tax situation. And all the data already exists. There's just no framework set up to allow this to be automated like there is in other countries.
It should be the case that all your basic taxes get calculated for you and taken at the point you're paid by your employer. Anything exceptional should be able to be claimed back via a web portal somewhere.
So it's not like 160m tax returns NEED to be filed. That's just how it is today.
There are a lot less loop holes than in the past. In the 1950s taxes on the rich were 90% - but there were so many loopholes the rich in reality paid a similar tax rate to their peers today where the tax rates are lower, but there are also less loopholes.
In the 1950s the common person couldn't take advantage of most loopholes (I'm not old enough to remember, but I'd guess mortgage interest was the only useful one, the rest where $100 here and there but it never added up to much for the common person)
Most of the tax code is irrelevant to 90+% of people. ~90% of people just claim the standard deduction every year, you have to be significantly well off or in an odd tax situation for itemized deductions to come out to more than the standard deduction.
The tax code is a behemoth. Plenty of loopholes to find to save money.
Also, most of the tax prep companies are thinly disguised payday loan companies.