I think it really depends what part of math and what part of computer science you're talking about. Certainly foundations of computation use finite objects from countable sets... But there is more to CS than turing machines and encodings of numbers.
For example: Computer vision, machine learning, data science, cryptography, etc are all rife with infinite objects!
Proof assistant software and SMT solvers can also prove things about infinite mathematical structures from number theory, ZFC, topology, etc.
Pure mathematics also cares about finite algorithms. Every proof is a finite sequence of deductions on finite objects from a countable set... eg a computer program!
Other examples include: computing bounds, integrals, roots of polynomials, divisors, bases, fundamental groups, etc. Pure math is full of computation!
tl;dr the line between math and cs is extremely fuzzy.
For example: Computer vision, machine learning, data science, cryptography, etc are all rife with infinite objects! Proof assistant software and SMT solvers can also prove things about infinite mathematical structures from number theory, ZFC, topology, etc.
Pure mathematics also cares about finite algorithms. Every proof is a finite sequence of deductions on finite objects from a countable set... eg a computer program! Other examples include: computing bounds, integrals, roots of polynomials, divisors, bases, fundamental groups, etc. Pure math is full of computation!
tl;dr the line between math and cs is extremely fuzzy.