Python generator and iterator

What is Iterable

An iterable is an object that can be iterated upon, meaning that you can traverse through all the values. Examples of iterable objects include lists, tuples, dictionaries, strings, and sets. In Python, an iterable object must implement the __iter__() method, which returns an iterator. This method is called when the iter() function is used on the object.

What is Iterator

An iterator is an object that represents a stream of data. It defines a method, __next__(), which is used to access the next item in the stream. The iterator also keeps track of the current position in the stream, so that it knows where to start the next iteration. When there are no more items to return, the __next__() method should raise a StopIteration exception.

What is Iteration

Iteration is the process of repeatedly extracting the next item from an iterator. It is often used in conjunction with loops, such as for loops or while loops. The for loop in Python, for example, automatically creates an iterator for the iterable object, and repeatedly calls the __next__() method until a StopIteration exception is raised. Iteration allows you to work with large data sets or infinite sequences without using up all of your memory, by only loading the next item when it is needed.

1itr = [5, 2, 3]
2
3# iterator from the list
4iterator = iter(itr)
5
6print(next(iterator))  # output: 5
7print(next(iterator))  # output: 2
8print(next(iterator))  # output: 3

values in list all iterated, try to do one more iteration, it will throws StopIteration exception

1print(next(iterator))  # output: StopIteration Exception

Custom iterator

To create a custom iterator, you need to define a class that implements the iterator protocol. The iterator protocol consists of two methods: __iter__() and __next__(). The __iter__() method should return the iterator object itself, and the __next__() method should return the next item in the sequence.

Here is an example of a custom iterator that iterates over a range of numbers:

1class CreateRange:
2    def __init__(self, start, end):
3        self.start = start
4        self.end = end
5        
6    def __iter__(self):
7        return self
8    
9    def __next__(self):
10        if self.start >= self.end:
11            raise StopIteration
12        current = self.start
13        self.start += 1
14        return current
15
16for i in CreateRange(0, 5):
17    print(i)

In this example, CreateRange is a class that implements the iterator protocol. The __iter__ method returns the iterator object itself, which is self. The __next__ method returns the next number in the range and increments the start value. The for loop iterates over the CreateRange object and prints out the numbers from 0 to 4.

1m = CreateRange(0, 3)
2print(next(m))  # output: 0
3print(next(m))  # output: 1
4print(next(m))  # output: 2
5print(next(m))  # output: StopIteration Exception

What is Generator

A generator in Python is a special type of iterator that allows you to iterate over a sequence of items, but without the need to store the entire sequence in memory. Instead, generators use a special type of function called a generator function, which uses the yield keyword to return items one at a time, as they are requested.

Normal function

In normal function statements after return will not execute, function will terminates after return

1def normal_function():
2    print("BEFORE - will execute")
3    return 1
4    print("AFTER - will not execute")
5
6normal_function() # output : BEFORE - will exceute

Generator function

A generator function is defined like a normal function, but instead of returning a value, it uses the yield keyword to return a generator object.

1def generator_function():
2    for i in range(5):
3        yield i

1generator_function() # output : <generator object generator_function at 0x000001B87C217190>

1for j in generator_function():
2    print(j)
3output :
40
51
62
73
84

Here is an another example of a simple generator function that generates the Fibonacci sequence:

1def fibonacci(n):
2    a, b = 0, 1
3    for i in range(n):
4        yield a
5        a, b = b, a + b
6
7for number in fibonacci(10):
8    print(number)

Here, the fibonacci function is a generator function that uses the yield keyword to return the next number in the Fibonacci sequence each time it's called. The for loop iterates over the generator object returned by the fibonacci function and prints out the numbers of the sequence.

The main advantage of using generators is that they allow you to work with large data sets or infinite sequences without using up all of your memory. They are also more efficient than lists because they only return one item at a time and don't need to store the entire sequence in memory. Generators can also be used to make the code more readable and to make it more memory efficient.