Random Python tricks: a short cheatsheet

Basic Python tricks

Inline if statement

Let’s get started with the simplest form. The multi-line if statement looks like this¹:

1
2
3
4


if condition:
   value_when_true
else:
   value_when_false

The one-liner goes like this:

1

value_when_true if condition else value_when_false

Let’s add another condition with an elif. The multi-line version looks like this:

1
2
3
4
5
6


if condition1:
    expr1
elif condition2:
    expr2
else:
    expr

To do it in one line:

1

expr1 if condition1 else expr2 if condition2 else expr

A multi-line nested if statement looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


if condition1:
    expr1
elif condition2:
    expr2
else:
    if condition3:
        expr3
    elif condition4:
        expr4
    else:
        expr5

And the solution in one line:

1

expr1 if condition1 else expr2 if condition2 else (expr3 if condition3 else expr4 if condition4 else expr5)

This kind of multi-nesting can be avoided with a refractor or with the smart use of dictionaries and lists. You can read more on refractors to reduce nesting in CodeAesthetic’s video.

Examples of * and ** unpack operator in python

How to use the ** unpack operator as entry for functions

This one is cool to follow the DRY principle. I use it for repeated arguments for multiple plots in Matplotlib. You can set parameters to a function with a dictionary:

1
2
3
4
5
6
7
8
9


def my_sum(a, b, c):
    return a + b + c

aux_dict = {"a": 1, "b": 2, "c": 3}

# you can use b and c
my_sum(a=1, **aux_dict)
# and reuse it for another function call
my_sum(a=2, **aux_dict)

Unpacking list with the * operator

You can use unpack inside of another list. And feed that into a function:

1
2


a = [1, 2, 3, 4, 5]
sum([1, 2, 3, *a])

You can also unpack the list as arguments of a function directly

1
2


b = ['oso', 'peludo']
print(*b) 

For further reading on the * python operator you can read W3schools , This Pointer, and Finxter

Get all methods of an object

Given an object foo use dir(foo) to print all its methods like this:

1

dir(foo)

Join multiple lines of a string into one long string

If you have a very long chain of characters, you can split it this way:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


# This
a = "This is a very long chain help I can't stop this is too much"
# Or this
a = ("This is a very long chain help "
"I can't stop this is too much"
"AAAAAAaaaAAAAaaAAAAaaAAAaaaaa")
# Also this
a = ("This is a very long chain help " 
+ "I can't stop this is too much" 
+ "AAAAAAaaaAAAAaaAAAAaaAAAaaaaa")

It can also be used to join multiple lines of text into a single one.

Importing modules from parent directories

Lets use this example:

1
2
3
4
5
6


.
├── dir1 (you are here)
└── dir2
    └── level1
        └── level2
            └── module.py

if you want to import modules inside ../level1/level2, you need to add that directory to the sys.path:

1
2


import os, sys
sys.path.append(os.path.join('..', 'level1', 'level2'))

Alternative and fast way of making loops

Using itertools, you can create a loop without creating auxiliary variables or casting an iterable:

1
2
3
4


import itertools

for _ in itertools.repeat(None, times):
    ...

Add a True/False switch flag to a command line Python script

A little cheatsheet on how argparse works. I tend to use docopt more now. But argparse is part of the python standard library so in some cases you may have no other option.

1
2
3
4
5
6
7
8


import argparse
parser = argparse.ArgumentParser(
    description="Flip a switch by setting a flag"
)
parser.add_argument('-w', action='store_true')

args = parser.parse_args()
print(args.w)

Usage example:

1
2


python myparser.py -w
>> True

Other example:

1
2
3
4
5
6
7
8


import argparse
parser = argparse.ArgumentParser('parser-name')
parser.add_argument(
    "-f", 
    "--flag", 
    action="store_true", 
    help="just a flag argument"
)

Usage example:

1

python3 script.py -f

If you ask script.py to return the value asociated to -f:

1
2


args = parser.parse_args()
print(args.f)

Output:

1

True

Some IPython magic tricks

1
2
3
4
5
6


# inside IPython you can use this to see all variables
%who 
# to compare time of execution of a line 
%timeit line_of_code
# run with debugger on a certain line
%run -d -bnline ex25.py

Ipdb: Python debugger

Here we have a few shortkeys for the Ipdb python debugger ²:

1
2


n[next] goes next line
s[tep] goes inside 

Conda

Create a conda enviroment given a name, an enviroment yaml config file and the python version.

1

conda env create --name thesis --file enviroment.yml python=3.8

Pandas

check if is view

Not so good practice, it is a private attribute

1

df2._is_view

Alternatively, comparing two DataFrames to determine if they have the same base.

1

df1.values.base is df2.values.base

Faster Pandas concatenation

Pandas is not optimized for dynamic concatenation. Instead of using a for loop like this one:

1
2
3
4


i = 0
while i < (max_count // int(counts[tag])):
    array = pandas.concat([array, group])
    i += 1

Use the appropiate pandas method concat instead:

1

pandas.concat([group] * (max_count // int(counts[tag])))

The same applies to appending new data to a pandas.DataFrame. It’s better to use the append method than to do it dynamically.

(Source: StackOverflow , 26/8/2021)

Some good libraries

docopt: Easiest way to create a CLI interface. Not fully POSIX. But it is blazing fast to implement.
Scrapy: An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
Python gpu:
- saturncloud
- rapidsAI
- modin or cudf
Differential equations
- scipy.odeint: Differential equations solver inside of scipy.
- JiTCDDE (just-in-time compilation for delay differential equations): Scipy-based module to solve Delay Differential Equations (DDE).
- diff equations: Basically, use generator -> use object -> search compact form.

NumPy

Be careful with NumPy

For a few hundred elements of data, it is actually faster to use a list (even more if it’s defined by comprehension), always test the difference for your particular application.

NumPy create array from iterator

1
2


iterable = (x * x for x in range(5))
np.fromiter(iterable, float)

Matplotlib and Seaborn

plotting backends

Matplotlib change font size

You can touch your matplotlibrc or do:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


SMALL_SIZE = int( 8 * 1.5)  # 12 # int( 8 * 1.5) # 
MEDIUM_SIZE = int(10 * 1.5) # 15 # int(10 * 1.5) # 
BIGGER_SIZE = int(12 * 1.5) # 18 # int(12 * 1.5) # 

plt.rc('font', size=SMALL_SIZE)          # controls default text sizes
plt.rc('axes', titlesize=SMALL_SIZE)     # fontsize of the axes title
plt.rc('axes', labelsize=MEDIUM_SIZE)    # fontsize of the x and y labels
plt.rc('xtick', labelsize=SMALL_SIZE)    # fontsize of the tick labels
plt.rc('ytick', labelsize=SMALL_SIZE)    # fontsize of the tick labels
plt.rc('legend', fontsize=SMALL_SIZE)    # legend fontsize
plt.rc('figure', titlesize=BIGGER_SIZE)  # fontsize of the figure title

Create a random image with a colormap and a random matrix

Let’s create a random image. We insert 256 rows with 256 distinct colors in random order. This is in order to not have repeated colors inside of a row.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


import numpy as np
import matplotlib.pyplot as plt

# an iterator that return a n_rows array of unique values 
# between 0 and n_rows -1. It does it a total of n_rows times
def color_array(n_rows):
    pick_from = np.arange(n_rows)
    for _ in range(n_rows):
        array = np.random.choice(
            pick_from, size=n_rows, replace=False
        )
        for val in array:
            yield val

n_rows = 256
shape = (n_rows, n_rows)
# Random image
n_filas = (n_rows)
itera_color = color_array(n_filas)
imagen = np.fromiter(
    itera_color, dtype=np.int64
).reshape(shape)

If you don’t care about repetitions inside the array, you can use this version:

1
2
3
4
5
6


# Create an array of random integers from 0 to n_rows-1
def random_int_array(n_rows):
    return np.random.randint(n_rows, size=(n_rows, n_rows), dtype=np.int64)

n_rows = 256
imagen_with_repetitions = random_int_array(n_rows)

Let’s display the image with no axis. I will save all images as SVG, but you can choose PNG, PDF or some compatible format:

1
2
3
4
5
6
7


fig1, ax1 = plt.subplots()
ax1 = plt.imshow(imagen)

# aesthetics
ax1.axes.axis('off')
aux_kargs = {'bbox_inches': 'tight', 'pad_inches': 0}
fig1.savefig("random_image.svg", **aux_kargs)

Image made with matplotlib. Created with a random data matrix and a colormap.

We can sort each row of the image this way (sorting horizontally):

1
2
3
4
5
6
7


# Sort columns
imagen_col_sorted = np.sort(imagen, axis=0)

fig2, ax2 = plt.subplots()
ax2 = plt.imshow(imagen_col_sorted)
ax2.axes.axis('off')
fig2.savefig("random_col_sorted.svg", **aux_kargs)

random_col_sorted — An image created with an array and a colormap, sorted by column in the array. Matplotlib Random image sorted by column

We can sort each column of the image this way (sorting vertically):

1
2
3
4
5
6
7


# Sort rows
imagen_row_sorted = np.sort(imagen, axis=1)

fig3, ax3 = plt.subplots()
ax3 = plt.imshow(imagen_row_sorted)
ax3.axes.axis('off')
fig3.savefig("random_row_sorted.svg", **aux_kargs)

random_row_sorted — An image created with an array and a colormap, sorted by rows in the array. Matplotlib Random image sorted by rows

Likewise, we can sort the array as a whole and reshape (every value in order):

1
2
3
4
5
6
7


# Sort flatten
imagen_flatten = np.sort(imagen, axis=None).reshape(shape)

fig4, ax4 = plt.subplots()
ax4 = plt.imshow(imagen_flatten)
ax4.axes.axis('off')
fig4.savefig("random_sort_flatten.svg", **aux_kargs)

random_sort_flatten — Numpy array, flatten, sorted and reshaped.

Large Matplotlib colormaps have white lines when saved as a file

You need to insert a True on the rasterized argument like this:

1

rasterized = True

Put the legend outside in a Matplotlib figure

First, let’s assign and tag the plots:

1
2
3
4
5
6
7
8


import matplotlib.pylab as plt

# The data
x =  [1, 2, 3]
all_y = [[1, 2, 4], [2, 4, 8], [3, 5, 14]]

# Labels to use for each curve
line_labels = ["Item Aasdfasdf", "Item B", "Item C"]

Then, assign the figures and titles:

1
2
3
4


fig, ax = plt.subplots()
ax.set_title(
    'Example of a Legend Being Placed Outside of Plot'
)

After that, plot all the curves. Set the legend, adjust the plot to the right and save.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


# plot and get the created line objects
l1 = ax.plot(x, all_y[0], color="red")[0]
l2 = ax.plot(x, all_y[1], color="green")[0]
l3 = ax.plot(x, all_y[2], color="blue")[0]

fig.legend(
    [l1, l2, l3],         # List of the line objects
    labels=line_labels,   # The labels for each line
    loc="center right",   # Position of the legend
    borderaxespad=0.1,    # Spacing near legend box
)

# Adjust the scaling factor to fit your 
# legend text completely outside the plot
# (smaller value results in more space 
# being made for the legend)
plt.subplots_adjust(right=0.75)

fig.savefig('image_output_leg.svg',
            bbox_inches='tight')

image_output_leg

You can also do it by adjusting bbox_to_anchor:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


ax.plot(x, all_y[0], color="red")
ax.plot(x, all_y[1], color="green")
ax.plot(x, all_y[2], color="blue")

# Add legend, title and axis labels
# adjust the x position of the starting point
# of the legend object so it can fit the figure
lgd = ax.legend(
    line_labels,
    loc='center right', 
    bbox_to_anchor=(1.4, 0.5) # x, y
) 

fig.savefig('image_output.svg',
            # bbox_extra_artists=(lgd,),
            bbox_inches='tight')

image_output

Matplotlib interactive inside IPython

To allow IPython to print plots in a new window use this:

1

plt.ion()

Or this:

1

%matplotlib

Iterating trough colors: Plotting curves following a colormap

Plotting curves with different colors in a colormap is an accesible task with Python. First, we split the [0, 1] interval in the amount of curves we are working with. Then we pick colors from a colormap in this case Gnuplot. In an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


import matplotlib.pyplot as plt

from matplotlib import cm
from numpy import linspace

start = 0.0
stop = 1.0
number_of_lines = 80
cm_subsection = linspace(start, stop, number_of_lines) 

colors = [ cm.gnuplot(x) for x in cm_subsection ]

fig, ax = plt.subplots()
for i, color in enumerate(colors):
    ax.axhline(i, color=color)

ax.set_ylabel('Line Number')
ax.set_ylim([0, ax.get_ylim()[1]])
aux_kargs = {'bbox_inches': 'tight', 'pad_inches': 0.1}
fig.savefig("line_numbers.svg", **aux_kargs)

line_numbers

Delete all lines in the upper right corner of the plot box: Axis spine deletion

A simple aesthetic change. You can change the Matplotlib config file, or use a function like this if you have the axis:

1
2
3
4
5


def axes_no_corner(ax):
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.get_xaxis().tick_bottom()
    ax.get_yaxis().tick_left()

This is a solution if you want to apply the solution to a single axis

I prefer the config file approach, that applies to every plot after is setted. You can see how I apply it in my other blog post.

Fix LaTeX Matplotlib errors

It can help to clear the cache ³:

1
2


import matplotlib as mpl
mpl.get_cachedir()

Source: Matplotlib Docs

Python If-Else One-Liner Examples. (n.d.). Retrieved April 7, 2023, from https://www.golinuxcloud.com/python-if-else-one-line/ . ↩︎
Wang, C. (2017, July 12). ipdb cheat sheet. Retrieved April 7, 2023, from http://wangchuan.github.io/coding/2017/07/12/ipdb-cheat-sheet.html ↩︎
Matplotlib Troubleshooting FAQ. (n.d.). Retrieved April 7, 2023, from https://matplotlib.org/stable/faq/troubleshooting_faq.html#locating-matplotlib-config-dir ↩︎