My party is a fan of dice throwing. A lot. And although they are not analysing things, they prefer visible improvements, like in numbers: proficiency bonus, ability scores, skill scores, etc.

Because of this, I'm considering of removing the 20 cap on ability scores, so they can get over 20 with ASI (exclusively), but usual means of bypassing the cap is also possible.

I'm aware of "bounded accuracy", but I don't see immediate danger of allowing this.

What will happen if I remove the cap? Will the game be broken?

Casters become much stronger

The largest problem I foresee is Save DC.

By RAW, Save DC is soft-capped at 19. 8 + Proficiency + Ability. Without the 20 cap, that goes to 24 or more, raising the save DC to 21, plus bonuses from magic items, specialized builds, etc.

This is a problem because Save DC is tied to a single ability score but can force saving throws for every ability score. Offenses will go up much faster than defenses, making everything bloodier.

Dexterity builds become much stronger

The other major issue I foresee is that this makes Dex builds the best melee builds. AC and Damage and Attack all from increasing Dex, with AC soft-capping around 22 for +3 Leather and a 24 Dexterity; compare that to the Strength build's AC 21 for +3 plate.

There are other issues, but those two are the ones that trouble me most.

Less character diversity as Feats become less attractive

When you get an ASI, often the most useful and powerful thing any character can do with it is boost their primary ability. Most players will not consider taking a Feat until their primary ability score is maxed at 20. By removing the cap, you make it so this "default-best-option" of raising your primary ability score is always available, and thus make players choosing Feats to make their characters more unique less likely.

Characters are more like glass cannons

In the same vein, now that there's no limit to your primary ability score, raising secondary ability scores for defense is much less attractive. Where before a Wizard with 20 INT has reached the limit and might consider raising DEX or CON to increase their longevity, now the siren song of higher Save DCs and most importantly an extra spell prepared per day will almost certainly cause them to raise INT instead - making them more powerful offensively at the cost of their defense.

This will make it harder for you to create good combats, as you try to find the knife-edge balance between "not any threat" and "kills them". (I have extensive experience with a highly offensively focused, low-defense party, and this measure would make that problem considerably worse)

Prepared spellcasters are more versatile than designed

ValhallaGH has already pointed out how spellcasters will become more powerful with high save DCs. As well as that, the number of spells clerics, druids, and wizards can prepare is tied to their casting stat. The game is designed so the bonus that stat provides to max out at 20 - if it can go higher, those classes are being given more versatility than intended (to the detriment of the niches of other classes, which may become overshadowed).

Lastly, a frame challenge: do your players really feel this way?

When I started playing D&D, I was highly focused on optimisation. I raised my ability scores with ASIs even when there were cool and interesting feats available, because I thought it was more optimal. I chose a less interesting subclass because I thought it was more mechanically powerful. And several levels in I realised I had been making all the wrong choices, because I had not built my character with any interesting mechanics available, and I was now bored with how my character played in combat, even though the numbers I was adding to my rolls were big. Perhaps your players really do only care about higher and higher numbers - but perhaps you/they just think they do.

DnD is a 'd20' game. And the main issue is that rolling a d20 against a target becomes much harder to balance when the modifiers that you add to a d20 roll become too large.

First, you will probably end up with a much larger disparity between characters such as Fighters who can, say, focus entirely on Strength to hit and characters who need other stats,e.g. Bards or Clerics, but still just need to hit things sometimes.

Weight that against an enemy's AC, which is not tied to a stat and is in a relatively small range of numbers.

You will get characters that virtually always hit, which removes any sense of competition or tension. This makes it tempting for a DM to artificially raise ACs to bring back some challenge for the Fighters, but then the Bards and Clerics etc stand no chance at all.

For a DM it also becomes much harder providing challenging but balanced encounters. Currently, since the numbers are 'bounded', its actually OK for an encounter to feature enemies of varying levels (something that in earlier editions was difficult without a lot of DM fudging precisely because they didn't have bounded accuracy). Even lower-level enemies can provide a small amount of danger and challenge en mass.

But remove that bound and allow modifiers for characters to go up as much as they like and the problems of earlier editions come back: A DM becomes very restricted in what monsters to put into an encounter that can actually be challenging without being a boring slaughterfest.

Link to Bounded Accuracy philosophy of 5th edition