Useful commands for IPython and Jupyter Notebooks
Photo by Greg Rakozy on Unsplash
When Python first appeared in 1991, it was seen more as an ?at your own risk? computing language. Today it is the predominant language for data science, machine learning, and software development.
A key reason behind Python?s popularity is its flexibility when it comes to adding new features and technologies like magic commands.
So, what exactly is a magic command in Python?
Magic commands are enhancements or shortcuts over the usual Python syntax designed to facilitate routine tasks.
These special commands enable us to easily control the behavior of the IPython system and solve various common problems in standard data analysis, for example, running an external script or calculating the execution time of a piece of code.
In this tutorial, I will teach you some very useful spells. These will help you to become better programming wizards.
All these commands were tested beforehand on a Jupyter Notebook running Python version 3.7.4. These commands should run perfectly in any IPython shell on your IDE, like Spyder, Pycharm, etc.
Magic commands come in two different forms:
- Line magic ? is denoted by a single % prefix and operates on a single line of input
- Cell magic ? is denoted by a double %% prefix and operates on the entire cell or multiple lines of input.
Let?s look at some of the most popular magic commands.
Run an External Script
Suppose you are working on a new project on Jupyter, and you wish to use a script that you wrote earlier. Rather than copying the entire code into a Jupyter cell, you can use some Python magic.
Any script can be run inside the environment of our IPython session using the %run command.
Suppose I have a file called test_script.py with the following code:
To run it, we use:
%run test_script.py
The script will run in an empty namespace with no imports or variables defined.
The behavior is identical to running the program on a command line using python test_script.py.
Note: You can provide access to already defined variables using %run -i.
All of the variables defined in the executed script are now accessible in the IPython shell.
For example, see how I reuse a function defined in the external script:
Source: Author
If needed, we can also pass arguments. Here, we pass a single integer argument along with our command.
%run python_script.py 10`
You can read more about running an external script here.
Import/View an External Script Into a Cell
We can run an external script using the %run command. But, there are situations when we must copy the script into a notebook cell.
Don?t worry, you don?t need to go back to Ctrl-C and Ctrl-V.
Photo by Elly Johnson on Unsplash
The %load magic function allows us to import a script directly into a code cell.
%load test_script.py
Just type the above statement and press run. Here, see the magic yourself:
Source: Author
Similar to the load command, we have pycat. It allows us to view the content of any file without leaving our notebook.
%pycat test_script.py
%pycat displays all the content of the external file as its output.
Source: Author
You can read more about importing an external script here.
Export a Notebook Cell as an External Script
As a data scientist, I often end up coding in a Jupyter notebook. But when it comes to production, the code needs to be in a .py script file.
Here, the writefile magic comes in handy. writefile exports the entirety of a cell?s content to an external file.
Take a look:
Notice the double percent(%%) sign here, as writefile is a cell magic.
We can also append code to an existing file.
You can read more about exporting a cell here.
Code Execution Time
%timeit calculates the time required by the IPython environment to execute a Python expression.
Photo by Aron Visuals on Unsplash
Let?s use this to compare the performance of list comprehension with a for loop.
We will create a list that contains the squares of the first 1,000 natural numbers.
Source: Author
timeit performs multiple runs of the expression. As we receive an average of several executions, the results are more reliable.
We also have a cell version for timeit (%%timeit) for a code block.
Let?s create the same list using a for loop:
In this scenario, list comprehension is about 10% faster than an equivalent for loop.
You can read more about code time execution here.
List Variables in the Namespace
If we need to list all the variables defined in the namespace, we have three commands at our disposal.
- who: Print all the interactive variables, with minimal formatting
- who_ls: Returns all the variables as a sorted list
- whos: Like %who, but gives some extra information about each variable
Here?s an example:
The above code displays all the user-defined variables in the namespace. We can display specific data types by passing them as arguments.
%who str# Output# name
who_ls works very similarly. The only difference is that the output is in the form of a list.
%who_ls#Output:# [‘age’, ‘name’, ‘roll_number’]
whos provides detailed information about the variables.
Source: Author
You can read more about listing variables here.
Execute HTML Script
If you need to provide some simple UI elements to your code, you can execute HTML and JavaScript code in IPython as well.
%%html allows us to write HTML code in the cell. For JavaScript, use %%js.
Here?s an example that will render a simple table in HTML:
The cell acts as an HTML editor. Once the code is run, you receive the HTML output.
Source: Author
You can read more about executing HTML script here.
Work With Environment Variables
The env magic command allows you to access and manipulate system environment variables.
Photo by Isaac Smith on Unsplash
You can either:
- %env ? List all the environment variables
- %env var ? Get value for variable var
- %env var value ? Set value for variable var
Without any parameter, %env will list down all the environment variables.
Source: Author
We add a parameter to return the value of a specific variable.
Source: Author
You can read more about working with environment variables here.
Conclusion
You can see a general description of available magic functions, along with some examples, using:
%magic
We can also get information on a specific magic function. So, for example, to read the documentation of the %who magic simply type this:
Source: Author
If you wish to define your own magic functions, you should read this: More IPython Resources.
So here were some of my treasured Python magic commands. Let me know your favorite ones in the comments below.
I hope you found this tutorial interesting and useful.
Further Reading
- Built-In Magic Commands IPython
- Tutorials Point ? Magic Commands
