Python – Benefits of using a Virtual Environment

A virtual environment can be extremely useful when developing multiple projects that require different dependencies, developing across multiple computers, and making it easier to collaborate with others on projects. In this post I will be outlining what exactly a virtual environment is and how to properly use it.

 

What is a Virtual Environment?

A virtual environment is simply a directory somewhere within your project files that contains all of the files and dependencies necessary to run python scripts that you leverage to run the project you are working on. There are more technical things that go on in the background that simply cloning the directory, but when you visually inspect the contents of that directory you will notice that it is very similar to the directory that you have python installed on in your system. What you then are able to do is install any external dependencies your project needs to run (like requests) into this virtual environment versus installing them to your global python install. This gives the benefit that if you need a specific version of a dependency like requests or a specific python interpreter you don’t need to un-install the version you have installed globally and then install the version your project calls for every time you switch projects.

 

How do i setup a virtual environment?

The only prerequisites you will need are a working Python install and pip installed

For this I am going to assume that you have a common directory you use to work on all of your projects for instance something like “/home/kyle/Documents/PythonProjects” and within that directory you keep all of your projects, we will also assume you have a project called “DataCruncher” and within that directory you keep all of your source files related to that project.

The tool you will use to create your virtual environments will be virtualenv and you can install it by issuing the following on the command line, note this will work in both Windows, Linux, and Mac although on Linux I find its easier to use the package manager integrated with the distro (ex. apt-get).

This will install virtualenv to your system and this should be the last time you need to use pip globally if you keep a virtual environment in all of your projects. Now what you will want to do is navigate to the root of your project, in this example it is “/home/kyle/Documents/PythonProjects/DataCruncher” and while in that directory issue the following command.

This will create a new virtual environment called ‘env’ you can name it anything you want it but ‘env’ seems to be used most places. One thing you will notice different between doing this on Windows vs a Unix like operating system such as Mac and Linux is the directory structures will be slightly different but it will not take long to locate your python.exe file along with pip.exe and all your other files.

If you have multiple python interpreters on your system you may need to specify which python interpreter to use when building your virtual environment, that can be done with the ‘-p’ flag, an example is shown below specifying to use a python 3 interpreter.

At this point you have a vanilla virtual environment setup and ready to go.

 

Configuring your virtual environment

If you have an existing project it is likely it has dependencies that you need to install that don’t come in the standard python library, a common example is ‘requests’. If you just type ‘pip install <packagename>’ that wont do you any good because it will use pip included in your global install of python, so what you need to do is locate pip.exe in the directory structure of your virtual environment previously created (in the above example ‘env’) and use that pip to install your dependencies. Below i have example commands to install requests in your virtual environment for Linux and Windows assuming we are still in the root directory of our project.

Linux:

Windows:

Again the command is the same the only difference is the path that the files are located.

 

Running your script with your virtual environment

Again we need to specify to use python.exe within our virtual environment vs our global install, I am assuming we are again in our project root directory and have “DataCruncher.py” file we want to run so to do this I provided example commands below for both Windows and Linux.
Windows:

Linux:

As long as your script executes properly and doesn’t throw a bunch of errors everything went well and you just ran your python file from your virtual environment.

 

Streamlining your virtual environment

Activating the environment:

As you probably noticed we had to specify the full path of pip.exe and python.exe when running them in our virtual environment, there is a tool that was provided during the build process of our virtual environment that will temporarily change some settings and allow us to just type “pip install <package>” or “python script.py” directly and not need to specify the full path of our interpreter inside the environment.

Again I have provide the commands to do this both in Windows and Linux.

Windows:

Linux:

Now that we see the “(env)” added to the beginning of our prompt we know that when we type “pip install requests” or “python DataCruncher.py” it will utilize the virtual environment and not the global install of Python on your system. Once you are done working on your project to get back out of your virtual environment on both Windows and Linux the command is the same.

And now that “(env)” is no longer in front of our prompt everything is back to normal and we are outside of the virtual environment

 

Specifying requirements:

One thing you can do is also create a ‘requirements.txt’ in the root of your project and put in the dependencies that your project needs, one way you can list all of the packages you have installed on your system is by issuing the following command.

This will be the exact syntax you will need to use in your ‘requirements.txt’ file to specify the package and version your project requires. Once you compile that list, instead of having to install all of the dependencies manually you can issue the following command from within an active virtual environment and it will install all the dependencies

You will then see the output of pip going out and grabbing all the dependencies and installing them in your virtual environment.

 

The Dont’s

One thing you will not want to do with your virtual environment is distribute it to other developers or machines via git or some other method. As I stated earlier there are a few technical things that happen when building the environment that tie it to the exact path on your system so you would need to make sure that you recreate it exactly on all systems you use it on. It would take more time to validate and troubleshoot all of that vs just having each developer create a new virtual environment for that project on their machine and with any modern version control system you can exclude the env directory from being uploaded to source.

 

Overview

Now that can be a lot to sink in and you may be thinking, that seems very complicated to add to my work flow. Let me present you with a scenario, you pull down someone else’s project either via git or some other method and they have a requirements.txt file with all the dependencies. You don’t have virtualenv installed, all you have is a fresh install of the Python version you need along with pip, and are at a terminal in the root of that project. Below are the following commands you will need to issue to build a virtual environment for that project.

At this point you can work to your hearts content on that project and when you are done all you need to do is isue the following command.

And from that point on all you will need to do to work within that virtual environment is the ‘source env/bin/activate’ before trying to run any code and ‘deactivate’ after your done working.

 

Thanks for reading this article, I hope you learned about the benefits brought by using a virtual environment and how easy it is to set one up and use. If there is anything here that is unclear or that you would like me to explain further feel free to leave a comment!

 

1 Trackback / Pingback

  1. HOW TO START WITH AI ? – AI TO ML

Leave a Reply

Your email address will not be published.


*