Learning to loop#
Looping over data is another fundamental programming concept. It’s necessary for a whole bunch of algorithms dealing with an amount of data larger then one. Sometimes its more than obvious, sometimes things happen behind the scenes. But using loops to work with data will almost always be there.
There are essentially two ways of looping over data in Python.
If we know the number of elements to work with in advance, we can use a for loop.
In all other situations, where data is consumed without the knowledge on its amout, we can use the while loop.
for loops#
To use a for loop the container with elements to loop over needs to support the iteration protocol, that is an __iter__ method exists. This is true for all the lists, tuples, sets and dictionaries we’ve seen earlier, and for generators.
Those generators are a special, memory efficient alternative to lists for example.
If the content of a list is known in advance, or in other words a plan exists how to create the content, than this plan can be brought to life using a generator concept, which will provide the desired content on the fly.
Without looking into the details on how to exactly do that, the range function is just that thing.
A generator to create numbers on the fly from a start to an end value with an optional setting for the step size.
The start value is optional here as well and has a default value of 0. Let’s use the help function again to have a closer look.
help(range)
Help on class range in module builtins:
class range(object)
| range(stop) -> range object
| range(start, stop[, step]) -> range object
|
| Return an object that produces a sequence of integers from start (inclusive)
| to stop (exclusive) by step. range(i, j) produces i, i+1, i+2, ..., j-1.
| start defaults to 0, and stop is omitted! range(4) produces 0, 1, 2, 3.
| These are exactly the valid indices for a list of 4 elements.
| When step is given, it specifies the increment (or decrement).
|
| Methods defined here:
|
| __bool__(self, /)
| True if self else False
|
| __contains__(self, key, /)
| Return key in self.
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __getitem__(self, key, /)
| Return self[key].
|
| __gt__(self, value, /)
| Return self>value.
|
| __hash__(self, /)
| Return hash(self).
|
| __iter__(self, /)
| Implement iter(self).
|
| __le__(self, value, /)
| Return self<=value.
|
| __len__(self, /)
| Return len(self).
|
| __lt__(self, value, /)
| Return self<value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __reduce__(...)
| Helper for pickle.
|
| __repr__(self, /)
| Return repr(self).
|
| __reversed__(...)
| Return a reverse iterator.
|
| count(...)
| rangeobject.count(value) -> integer -- return number of occurrences of value
|
| index(...)
| rangeobject.index(value) -> integer -- return index of value.
| Raise ValueError if the value is not present.
|
| ----------------------------------------------------------------------
| Static methods defined here:
|
| __new__(*args, **kwargs)
| Create and return a new object. See help(type) for accurate signature.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| start
|
| step
|
| stop
We can see the syntax to be used here, with the two options of calling and an optional step setting, as well as the necessary __iter__ method which essentially makes range a generator. And when trying to create numbers from 0 to 10 using range, we see, that just calling it won’t deliver the expected result.
range(10)
range(0, 10)
But as soon as we iterate over the generator we receive the numbers we are looking for.
for index in range(10):
print(index)
0
1
2
3
4
5
6
7
8
9
while loops#
The other type for iterating in Pythin is the while loop. It’s used whenever the amount of iterations to be done is not known in advance, such as when working with live data from some other application for example, or when reading a sensors output. The idea here is to keep on iterating whilst some condition is true.
Here’s a trivial example to mimic the behaviour of the for loop providing numbers from 0 to 10.
index = 0
while True:
print(index)
if index >= 9:
break
index += 1
0
1
2
3
4
5
6
7
8
9
Container types revisited#
We’ve learned about container types to organize Python objects in a previous chapter. There had been two ways differing in syntax to create the container type.
One long expression using the respective constructor calls, such as list(), tuple(), set() and dict(), as well as the short syntax using [], () and {}
We’ve seen how they are created being empty at the beginning and filled afterwards, or pre-initialized directly at definition time.
There is another, very elegant way to programmatically pre-initialize containers, at least for lists, tuples and dictionaries, closely modeled after the mathematical set building syntax.
Such as “a set of square numbers, such that the number is in the range 0 to 10 exclusively”,
or mathemetically written as
$${n^2|; n \in \mathbb{N}, 0 \le n \lt 10}$$
List comprehension#
The Python syntax for the explicit call is this
list(x**2 for x in range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
The shorter, much preferred version is written as
[x**2 for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
It can even be combined with an if clause to narrow the result set down. Let’s create a list of square numbers built from even numbers only this time to illustrate this.
[x**2 for x in range(10) if x % 2 == 0]
[0, 4, 16, 36, 64]
This is much more readable and enjoyable than the classic variant I think!
square_numbers = []
for x in range(10):
if x % 2 == 0:
square_numbers.append(x**2)
square_numbers
[0, 4, 16, 36, 64]
Tuple comprehension#
So that was the syntax for lists. What about the other container types. What about tuples?
tuple(x**2 for x in range(10))
(0, 1, 4, 9, 16, 25, 36, 49, 64, 81)
This created a pre-filled tuple with the desired square numbers. Matching the nature of a tuple, the contents of this tuple can be iterated over, but not altered in any way.
And what about the short syntax?
(x**2 for x in range(10))
<generator object <genexpr> at 0x7f2848156cf0>
Interesting! It did not really create a tuple with our squares right away, but something a lot more exciting.
The result is a generator, with the same usage pattern as the range function.
We have to iterate over the contents of the generator to receive our square values, which looks a bit weird in this very context.
for square in (x**2 for x in range(10)):
print(square)
0
1
4
9
16
25
36
49
64
81
Yeah. We’ve seen simpler ways before.
Set comprehension#
Next in the row is the set comprehension. It totally works as expected. Creating a set of squares just takes this line.
set(x**2 for x in range(10))
{0, 1, 4, 9, 16, 25, 36, 49, 64, 81}
Or this line for the shorter syntax.
{x**2 for x in range(10)}
{0, 1, 4, 9, 16, 25, 36, 49, 64, 81}
Dict comprehension#
Last in the row is our dictionary type. As it’s a key-value store, the syntax is slightly different. To create a number to square mapping, it takes this.
dict((x, x**2) for x in range(10))
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
The dict is created using a row of key-value tuples in this syntax. Much more preferred and more readable is the short syntax to create the dict.
{x: x**2 for x in range(10)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
Iterating over containers#
Now that we’ve seen how to basically use for and while loops and how to create containers in a very elegant and readable fashion, we should practice the use.
Let’s start with simply iterating over a list of well chosen numbers.
l = [1, 5, 3, 7, 2, 9, 2, 3, 4, 2]
The classic approach in most programming languages would be like this.
for index in range(len(l)):
print(l[index])
1
5
3
7
2
9
2
3
4
2
This works in Python as well. A much cleaner approach is this though.
for item in l:
print(item)
1
5
3
7
2
9
2
3
4
2
Here, we don’t make use of the index variable for accessing the numbers in the list, as the list type directly supports iteration.
If however the index of each item would be necessary for processing the items it can be generated on the fly using a wrapping enumerate call.
for index, item in enumerate(l):
print(f"Item {item} is at index {index}.")
Item 1 is at index 0.
Item 5 is at index 1.
Item 3 is at index 2.
Item 7 is at index 3.
Item 2 is at index 4.
Item 9 is at index 5.
Item 2 is at index 6.
Item 3 is at index 7.
Item 4 is at index 8.
Item 2 is at index 9.
This creates a tuple for each item in the list of numbers having the index and the number itself which are assigned to the variables index, and item in each iteration.
This is based on quite a wondeful and powerful concept in Python called variable expansion. Let’s have a closer look. Imagine you managed a number of color definitions in a list.
colors = [
(255, 0, 0), # red
(0, 255, 0), # green
(0, 0, 255), # blue
(255, 0, 255), # violet
(255, 255, 0) # yellow
]
If you wanted to assign these values to dedicated variables you could do this in a very classic manner like this.
red = colors[0]
green = colors[1]
blue = colors[2]
violet = colors[3]
yellow = colors[4]
Or, more pythonic, using variable expansion:
red, green, blue, violet, yellow = colors
And even more magically, if you just cared about green an blue
_, green, blue, *_ = colors
t = tuple(l)
for item in t:
print(item)
1
5
3
7
2
9
2
3
4
2
s = set(l)
for item in s:
print(item)
1
2
3
4
5
7
9
d = {
"time": 0,
"value": 0.1,
}
for key in d.keys():
print(key)
time
value
for value in d.values():
print(value)
0
0.1
for key, value in d.items():
print(f"The value for key {key} is {value}.")
The value for key time is 0.
The value for key value is 0.1.
for index, (key, value) in enumerate(d.items()):
print(f"The key-value pair {key}, {value} is stored at index position {index}.")
The key-value pair time, 0 is stored at index position 0.
The key-value pair value, 0.1 is stored at index position 1.
Creating a reverse lookup dict.
r_d = {value: key for key, value in d.items()}
r_d
{0: 'time', 0.1: 'value'}
You can iterate over text as well.
message = "Hello World!"
for char in message:
print(char)
H
e
l
l
o
W
o
r
l
d
!
all_names_str = "John Doe, Jane Here, Jack There, Rudi Brave, June Wright"
all_names = [name.strip() for name in all_names_str.split(",")]
fornames = []
surnames = []
for name in all_names:
forname, surname = name.split(" ")
fornames.append(forname)
surnames.append(surname)
print(fornames)
print(surnames)
['John', 'Jane', 'Jack', 'Rudi', 'June']
['Doe', 'Here', 'There', 'Brave', 'Wright']
all_names = []
for forname, surname in zip(fornames, surnames):
print(forname, surname)
all_names.append(f"{forname} {surname}")
print(all_names)
all_names_str = ", ".join(all_names)
print(all_names_str)
John Doe
Jane Here
Jack There
Rudi Brave
June Wright
['John Doe', 'Jane Here', 'Jack There', 'Rudi Brave', 'June Wright']
John Doe, Jane Here, Jack There, Rudi Brave, June Wright
", ".join([f"{forname} {surname}" for forname, surname in zip(fornames, surnames)])
'John Doe, Jane Here, Jack There, Rudi Brave, June Wright'