Infinite iterations are obtained either with an infinite iterator, with a while loop, or by recursion. Obviously, in practical cases, some condition stops the iteration. The difference with finite iterations is that it is impossible to say by a cursory examination of the code, whether the iteration will stop or not.
The while loop may be used to repeat a code block until a condition is fulfilled:
while condition: <code>
A while loop is equivalent to the following code:
for iteration in itertools.count(): if not condition: break <code>
So a while loop is equivalent to an infinite iterator, which might be stopped if a condition is fulfilled. The danger of such a construction is obvious: the code may be trapped in an infinite loop if the condition is never fulfilled.
The problem in scientific computing is that one is not always sure that an algorithm will converge. Newton iteration, for instance, might not converge...