If you need python libraries installed that we don't have, or that are newer than the versions we currently have installed, you should be able to install everything you need on your own using a virtual environment.
Here are the steps needed to set up your own python virtualenv:
All of these assume your CS username is
hpotter1 (i.e., when you execute the commands below, please change
hpotter1 to your username).
Also, all of these are if you haven't already done this. If you have, you can skip down to the next section (make the virtual env).
Fall2018 Note: all new accounts as of Fall 2018 already have these preliminary steps done, so you can skip to the next section.
Make a directory in
/scratch/hpotter1 for your venvs, so you don't go over your disk quota. If you use
/local instead of
/scratch, it will be faster, but local to the machine you are on (i.e., only accessible from that lab machine). Either works. Here are the commands for using scratch (again, assuming you are hpotter1):
mkdir /scratch/hpotter1/Envs ln -s /scratch/hpotter1/Envs ~/Envs
If you already have a
~/.cache/pip directory, move it to
/scratch/hpotter1 and link it back. If not, just make one in
/scratch/hpotter1 and link it back. Here are the commands:
mv ~/.cache/pip /scratch/hpotter1 # if you already have one, mv ln -s /scratch/hpotter1/pip ~/.cache/pip # and link it back
mkdir /scratch/hpotter1/pip # if not, make one ln -s /scratch/hpotter1/pip ~/.cache/pip # and link it back
Here's how to make a virtual environment, called
my_project, that uses python 3:
export PYTHONPATH= export WORKON_HOME=~/Envs source /usr/local/bin/virtualenvwrapper.sh mkvirtualenv -p /usr/bin/python3 my_project
At this point you have made and are in the virtual environment. Your prompt should have changed to include
(my_project) at the beginning.
pip to install any packages you need! Use
pip list to see what's installed. Here are some examples:
(my_project) pip install numpy (my_project) pip install scipy (my_project) pip install scikit-learn (my_project) pip list Package Version ---------------- ------- numpy 1.12.1 scikit-learn 0.18.1 scipy 0.19.0 (my_project) python >>> import sklearn >>> sklearn.__version__ '0.18.1'
At this point your virtualenv is ready to use. If you want, start using it and run your programs in it.
deactivateto end the virtualenv
If you want to get back out of your virtualenv, you can deactivate it or just close/exit the terminal window.
(my_project) deactivate $
To get back into your virtualenv:
$ export PYTHONPATH= $ export WORKON_HOME=~/Envs $ source /usr/local/bin/virtualenvwrapper.sh $ workon my_project (my_project)
I sometimes put the above four commands in a file, so I just have to
source that one file to get into my virtual environment (I also added some CUDA environment variables, in case you're using the GPUs):
$ cat goproj export CUDA_HOME=/usr/local/cuda-9.0 export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64 export PYTHONPATH= export WORKON_HOME=/scratch/hpotter1/Envs source /usr/local/bin/virtualenvwrapper.sh workon my_project $ source goproj (my_project)
If you want to share your virtualenv with a lab partner, you just need to allow read-execute access to the
Envs directory and any files below that. All that's stored in there is the software you installed (e.g., with
pip), so it's OK to open up this directory for others to access. Don't put your lab work in the
Here's a command to allow read and execute permission for everyone on the
/scratch/hpotter1/Envs directory, and any files and directories below that:
$ chmod -R a+rX /scratch/hpotter1/Envs
R is for recursive, and the
a+rX says grant all users read and execute access.
Here are a few pages with additional info on using python virtual environments:
Back to SwatCS Help Docs