Introduction#
This article will show you how to setup a Python development environment for AI. We will go through installing the python version that you need, a version manager, a package manager, a code editor, and a notebook environment. By the end of this article, you will have a fully functional Python development environment for AI (or any other python applications).
Installing a Package Manager#
If you are on a Mac, I insist that you install Homebrew. It is a package manager for Mac. If you are not on a Mac, you can skip this section.
Homebrew will make your life easier. Think of it as the App Store for your Mac that allows you to install applications from the command line with a simple command and have them organized.
It is a thousand times better than downloading applications from the internet and installing them manually.
Actually, sometimes it is the only way to install some applications. Right here, we will use it to install some of the tools that we need super easily.
So first let’s install Homebrew. Open your terminal and run the following command.
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
This command comes directly from the Homebrew website and it will install brew on your Mac. It will ask you for your password. Enter it and wait for the installation to finish.
Install Python in your System#
If you are on a Mac, you already have Python installed. You can check the version by running the following command in your terminal.
python --version
If you have never installed Python before, you will see that you have Python 2.7 installed. This is the default version of Python that comes with Mac, but it is an old version that is not supported anymore. We will install the latest version of Python in our system.
To do that, we will use brew. Run the following command in your terminal.
brew install python
This will install the latest version of Python in your system. You can check the version by running the following command in your terminal.
python --version
You should see that you have Python 3 installed. Great! Now we have Python 3 installed in our system. But we are not going to use it for development.
We will use it to run some of our applications that require Python. In case it is not clear, we will not use the system version of python for development.
Actually, using it is estimated that using the system Python for development is the root cause of 97% of software developer suicides.
We will instead use a Python version manager to install multiple versions of Python on our system and switch between them easily. Not only is it more professional, but it is also way easier.
Python Version Manager#
Great, now we have Python 3 installed in our system. This is what we call the system Python. It is the Python that comes with your system. It is the Python that you will use to run some of your applications that require Python. But it is not the Python that you will use to develop your applications!
Why do we need a Python Version Manager?#
Simple answer: To remove most of our headaches.
When you develop applications, sometimes you need to use a specific version of Python. For example, you might need to use Python 3.6 for one application and Python 3.8 for another application. You can’t use the system Python for that. In fact, you should never use the system Python for development!
Also, when you are a beginner in this, installing Python and creating a development environment can be a headache. By using a version manager, you can install one or multiple versions of Python on your system and switch between them easily. It takes a single command.
By having a Python version manager, you will be able to:
- Install multiple versions of Python on your system
- Switch between Python versions easily
- Create virtual environments for your projects
- Install packages in your virtual environments
- You can install multiple versions of Python on your system and switch between them easily.
It will also allow you to
- Not use
sudo(which is not recommended). - Not worry about messing up your system, projects or other Python versions.
Installing a Python Version Manager#
Think of a Python version manager as a snake master that will take care of all your snakes (your pythons). It will feed them, clean them, and make sure that they are healthy. This is what a Python version manager does, but with Python versions.
So let’s install a Python version manager. We will use pyenv.
Pyenv is a simple Python version manager that allows you to install multiple versions of Python on your system and switch between them easily. It is also very easy to install using Homebrew.
To install a package in Homebrew, you use the brew install command followed by the name of the package. So let’s install pyenv by running the following command in your terminal.
brew update
brew install pyenv
To check that pyenv is installed, run the following command in your terminal.
pyenv --version
Installing Python Versions#
Now your snake master is working. In other words, you have Pyenv installed on your system.
We can now install multiple versions of Python on our system and your snake master will take care of them and allow you to switch between them easily.
Let’s install Python 3.6 and Python 3.8. To do that, we will use the pyenv install command followed by the version of Python that we want to install. Let’s install Python 3.6 and Python 3.8:
pyenv install 3.6
pyenv install 3.8
This will install Python 3.6 and Python 3.8 on your system. You can check that they are installed by running the following command in your terminal.
pyenv versions
You should see that you have Python 3.6 and Python 3.8 installed. You should also see the system Python (2.7) in the list with an asterisk next to it. This means that it is the current version of Python that you are using.
But remember that we will not use the system Python for development.
Setting the Global Python Version#
Now let’s set the global version of Python to one of the versions that we installed.
But first, let me show you where your current Python is installed. Run the following command in your terminal.
which python
You should see that your current Python is installed in the /usr/bin/python directory. This is the system Python. As we said, we will not use it for development. So let’s set one of our installed versions as the global version of Python.
This will make it so that when we run the python command in our terminal, it will use one of our installed versions of Python instead of the system Python.
To do that, we will use the pyenv global command followed by the version of Python that we want to use globally. Run the following command in your terminal.
pyenv global 3.8
This will set Python 3.8 as the global version of Python. You can check that it is set by running the following command in your terminal.
pyenv versions
You should see that Python 3.8 is set as the global version of Python. Good job! You now have a Python version manager installed and you have multiple versions of Python installed on your system. You also have a stable version of Python set as the global version of Python.
Now, if you run the python command in your terminal, it will use the global version of Python that you set. You can check that by running the following command in your terminal.
python --version
And this global version is being executed from pyenv’s version. You can see that by running the following command in your terminal again.
which python
This will tell you the path to the Python executable that is being used. You should see that it is being executed from pyenv’s version.
Creating a Python Virtual Environment#
Great! Bravo for getting this far. Now you have a working version of python set as your global version. And it is being executed from pyenv’s –your snake master– version.
Now let’s start developing our applications. To do that, we will create a Python virtual environment.
Why do we need a Python Virtual Environment?#
Have you ever started a project and then realized that you need to install a bunch of packages? So you install them globally by running pip install my-package. That package will remain installed on your system even if you delete your project. And if you install another package with the same name, you might run into conflicts.
And what if you need to use a specific version of a package for one project and another version for another project? You can’t do that if you install them globally. Or what if you need a package that is not compatible with the version of Python that you are using? You can’t do that if you install them globally.
This is why we need a Python virtual environment. Here are some of the benefits of using a Python virtual environment:
It allows you to:
- Install packages locally for a specific project.
- Install different versions of packages for different projects.
- Install packages that are not compatible with the version of Python that you are using.
- Install packages without having to worry about conflicts.
- Bundle your application with the version of Python that you used to develop it and all the packages that you installed for that application.
If you are familiar with web development, initializing a Python virtual environment is similar (but not precisely the same) to initializing a NPM project. You are essentially creating a folder that contains all that you need to run your application.
Setting the Local Python Version#
Before we create a Python virtual environment, let’s select the python version that we want to use for our project. You may want to do this if some of the packages that you need for your application are not compatible with the global version of Python that you set.
If you want to use the global version of Python (the one that we set earlier), you can skip this step.
To set the local version of Python, we will use the pyenv local command followed by the version of Python that we want to use locally. Run the following command in your terminal.
pyenv local 3.8
This will set Python 3.8 as the local version of Python. You can check that it is set by running the following command in your terminal.
pyenv versions
You should see that Python 3.8 is set as the local version of Python.
Once you set the local version of Python, you can create a Python virtual environment that will use that version of Python.
Creating a Python Virtual Environment#
To create a Python virtual environment, we will use the native python command python -m venv followed by the name of the folder that we want to create. Run the following command in your terminal.
python -m venv my-venv
This will create a folder called my-venv in your current directory. This folder will contain a specific version of Python and all the packages that you install in that environment. If you selected a local version of Python, it will use that version. If you did not select a local version of Python, it will use the global version of Python.
Activating/Deactivating a Python Virtual Environment#
Great. But that itself does not change anything. In order to use it, we need to activate it.
To activate a Python virtual environment, we need to run the source command followed by the path to the activate file in the virtual environment. Run the following command in your terminal.
source my-venv/bin/activate
If you are using Windows, you will need to run the following command instead.
my-venv\Scripts\activate
This will activate the Python virtual environment. You can check that it is activated by running the following command in your terminal.
which python
You should see that it is being executed from the Python virtual environment that you created. You should also see that the name of the virtual environment is displayed in parentheses next to the path to the Python executable. This means that the Python virtual environment is activated.
From now on, every time you run the python command in your terminal, it will use the version of Python that is in that folder. And every time you run the pip command in your terminal, it will install packages in that folder (as long as you have activated the Python virtual environment).
To deactivate the Python virtual environment, you can run the following command in your terminal.
deactivate
Great! This is a huge milestone. You now have a Python virtual environment that you can use to develop your applications. Now you can install packages using pip and they will be installed locally, in that folder. Also, whenever you use the python command, it will use the version of Python that is in that folder.
Now you can focus on developing your application without having to worry about conflicts or python versions.
Using git to Manage Your Project#
This is more of a side note. But I think it is important to mention it. If you are not familiar with git, I recommend that you learn it. It is a version control system that allows you to keep track of your changes and collaborate with other developers.
Git itself can be quite complex, but you don’t need to know everything about it to use it. You can check out this article to learn the basics of git.
In order to install git, you can run the following command in your terminal (again using our great package manager brew).
brew install git
Once you have git installed, go to your project directory and run the following command in your terminal.
git init
This will initialize a git repository in your project directory. Now you can start tracking your changes.
Now add a .gitignore file to your project directory. This file will tell git to ignore certain files and folders. For example, you don’t want to track your Python virtual environment or other files that are not relevant to your project (or are secret, like your API keys).
To be safely covered, you can copy the .gitignore from this repository. It is a very generic .gitignore file for Python projects that will have you covered in most cases. If you want to add more files or folders to ignore, you can do that by adding them to the .gitignore file.
Finally, let’s just stage all the files in your project directory and commit them. Run the following commands in your terminal.
git add .
git commit -m "Initial commit"
This will stage all the files in your project directory and commit them. Now you can start tracking your changes.
Notebooks#
Notebooks are a great tool for data science and machine learning. They allow you to write and execute Python code in your browser. They are also great for sharing your work with others.
I mostly use them for skecthing on the side of my project, or performing exploratory data analysis and prototyping. Once I have a working prototype, I move your code to a Python script and run it from the command line.
Installing Jupyter Notebooks#
To install Jupyter Notebooks, you can run the following command in your terminal.
pip install jupyter
This will install Jupyter Notebooks in your Python virtual environment. Now you can run the following command in your terminal to start a Jupyter Notebook server.
jupyter notebook
This will start a Jupyter Notebook server in your terminal. You can now open your browser and go to http://localhost:8888 to access the Jupyter Notebook server. You can also click on the link that is displayed in your terminal.
Using Google Colab#
Google Colab is a great tool for running Jupyter notebooks in the cloud. It is free and it allows you to run your notebooks on a GPU or TPU. It is great if you are working with deep learning models.
To use Google Colab, you need to have a Google account. Just go to Google Colab and sign in with your Google account. Then you can create a new notebook or upload an existing one.
You can install packages directly from the notebook. Just run the following command in a cell.
!pip install package-name
Whenever you want to run a command that you would normally run in your terminal, you can just add an exclamation mark at the beginning of the cell.
What I like to do is to use Google Colab for my notebooks. It is a free service that allows you to run Jupyter notebooks in the cloud. It is great because you don’t have to worry about installing anything. You can just open a notebook and start coding.
Conclusion#
Congratulations! You have set up your Python development environment. You have installed Python, set the global version of Python, installed a Python virtual environment, and initialized a git repository. Now you can start developing your Python applications.
This may look a bit overwhelming at first, but it is actually quite simple. Once you get used to it, you will be able to set up your Python development environment in a matter of minutes since you will already have everything installed.
Good luck with your Python journey!