Lab 1: Image Processing
Due Monday, 14 September 2020, before 7am
This is an individual lab assignment. There will be an opportunity for partnered labs soon.
1. Lab Goals
Through this lab you will learn to
Setup and use tools needed throughout the semester
Manipulate images at the pixel level
Program a WebGL fragment shader
Recognize the parallel nature of shader programming
3. Cloning Files
As with all lab assignments this semester, you will use
git, a version control system, to get starting files as well as handing in your assignments. When you start an assignment, you’ll execute a
git command to grab some starting files. Similarly, when you want to submit an assignment, you’ll execute a couple of
git commands. Later on in the course, you’ll be using
git to facilitate paired lab assignments.
Since we are starting a new course, there is some first time setup required. The quick instructions below assume that you have access to your Swarthmore GitHub account and are logged into a terminal on the Swarthmore CS network via
ssh. Please consult the Remote Tools help if you need help connecting with
First, we will make directory for all our labs and clone
lab1 from Github CS40-F20 org
cd mkdir -p cs40/labs #make your labs dir cd cs40/labs pwd # should list /home/you/cs40/labs git clone email@example.com:CS40-F20/lab1-you
Longer GitHub setup instructions are available off the Remote Tools pages if you need help.
3.1. Setting a Symbolic Link
We will be starting with some low level graphics work in WebGL2, but we will not be writing everything from scratch. Much of graphics programming builds off of various third party libraries. We will use several of those throughout the course of this semester. Instead of every student installing a separate copy of these libraries, I have installed them centrally, and you can set a symbolic link to them. We’ll start by setting one link to a path that is hard to remember. Your git repos will have automatically contain relative symbolic links to the parent link for each lab.
cd cd cs40/labs #make sure you are in labs, not lab1 pwd # should list /home/you/cs40/labs ln -s /usr/local/stow/CS40-F20/lib/cs40lib/
3.2. Fixing a Symbolic Link
The original started code contained a bad link/file. You will need to fix this prior to getting the lab to work.
cd cd cs40/labs #make sure you are in labs, not lab1 cd lab1-you #use your username rm lib ln -s ../cs40lib ./lib #pay attention to the dots
4. Running and Viewing your Lab
This will require that you have a browser with WebGL2 support. Please visit the WebGL2 Check site and verify that you can see the spinning cube on your browser. WebGL2 can be used on most browsers, though support on Safari is "experimental". Try using an up to date version of Edge, Chrome, or Firefox.
Assuming your browser support WebGL2, we are ready to use your browser to view your code that you cloned on the CS machines. For somewhat technical reasons, the lab code must be run through a webserver. You cannot just open the files locally using
~public_html/ folder on CS, because then everyone can see your solutions.
Our approach for remote development will be to launch a small webserver locally on the Swarthmore CS machines and create a tunnel that provides a temporary URL that you can use to view your files remotely. This will allow you to view and test your code, without others being able to easily guess the URL and viewing your solutions without your knowledge.
4.1. python3 -m http.server
We will use a
python3 http server to start the service locally on Swarthmore machines, and
ngrok to set up the tunnel.
To run the server, change to the directory you want to serve and run the following command using
python3. Note the
& at the end to run the server in the background.
cd cd cs40/labs/lab1 python3 -m http.server &
You should get a message saying
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)
If instead you get a long error message ending in
OSError: [Errno 98] Address already in use
It may mean someone else is running a server on the same port, possibly you. If it is you, you can either continue using the existing server, or kill the old server using
You can also try running your server on a different port other than 8000
cd cd cs40/labs/lab1 python3 -m http.server 8080 & Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/)
As long as we eventually get a
Serving HTTP message, we are ready to proceed to the next step
4.2. ngrok http 8000
http://0.0.0.0:8080/ is only usable if you are on campus and logged in directly to the machine you started the server on.
ngrok tool will create a temporary public URL that you can use to access your webserver from anywhere. To run it, just provide the protocol and port number that your webserver is running on.
ngrok http 8000
This should pop up a small display in your terminal listing a Forwarding URL.
The session will last at most 8 hours, or until you stop the process with CTRL-C
You should now be able to go to the link given by
ngrok in a browser on your personal computer at home. If everything works, you should see the lab1 start page.
5. Making Modifications
The goal for this lab is to use programmable fragment shaders to process a sample image using WebGL2. Since we don’t know that much yet about any of these topics, most of the boilerplate has been provided and you can skip much of the code provided.
If you make modifications to the files on the CS server and save them, you should be able to refresh the browser and see your changes without needing to restart the servers or ngrok tunnel. If it appears your changes are not visible in your browser, you can try a hard refresh.
6. Add Split
The split filter should split the original image into four mini images, with each image only showing one color channel; red, blue, green, and greyscale. The greyscale image is created by averaging the values of the red, green, and blue components of a given pixel.
With the starter code provided, the split button simply creates a solid red image. Your output should be similar to the image below.
You can implement this effect by modifying the file
//export default `
but you’ll have to uncomment it before saving and running your program.
Each fragment shader has a single
main() function whose job is to compute and set a single output variable that is the color for a particular pixel. In this context, that output variable has the name
outColor and type
vec4 meaning it has four components, a red, green, blue, and alpha channel. For this week, and for several more weeks, you can safely ignore the alpha channel and always set it to
1. You can access the individual components of a
vec4 variable using the accessors
.r, .g, .b and
.a. These values are in the range 0.0 to 1.0, with white being
vec4(1,1,1,1) and black being
float red = outColor.r;
6.1.1. Shader Parallelism
a single call to
main() only processes a single pixel. OpenGL will call the shader multiple times for different pixels in parallel. This happens transparently behind the scenes without you needing to worry about it. You only need to worry about how to process one pixel.
Each time a fragment shader is called it may have different input variables. In this example, the only input variable is
in vec2 v_texCoord;
which describes the relative position of the pixel across the image. You can access the individual components of
.y, both of which are in the range
1. for this lab.
We can use these
v_texCoord coordinates to sample the provided image using
outColor = texture(u_image, v_texCoord);
and in fact, this is what is done for the
fs_pass.js file connected to the
Original button. Each time the shader is called,
v_texCoord has a different value and a new color is computed at that position. When the shader is called for all pixels in the canvas, a copy of the original image is produced.
6.1.2. Modifying the Shader
Your first goal is to modify the body of
fs_split.js to compute the correct value of
outColor to implement the split effect. You can create additional helper variables, manipulate vectors, and tweak components as needed. It won’t be a single one liner, but it isn’t that long either.
Here are some examples that while wrong, might help you understand how the variables work.
outColor = texture(u_image, 0.5*v_texCoord);
vec2 offset = vec2(0.5, 0.5); outColor = texture(u_image, v_texCoord+offset);
/* does not use colors from image at all */ outColor = vec4(v_texCoord.x, v_texCoord.y, 0., 1.);
Remember, the program will automatically run
main() for you multiple times on each pixel, you do not need to write any nested for loops to process each row/column.
7. Create Two New Effects
With one effect complete, be creative and add two more effects of your own choosing. Do not modify
fs_split.js to implement your effects. Instead copy
fs_pass.js to two new files and make your changes there, similar to how you modified
fs_split. Your effects should be a bit more elaborate than the ones you may have done in a similar lab for CS35
In addition to adding new shaders, there are a couple other places where you must modify code, but these changes are relatively small.
7.1. Adding Buttons
You need new buttons for your new effects. This is done in
index.html. Copy and paste the line
<button type="button" id="split">Split Effect</button>
And give each effect an new name and unique id. When you refresh the page, the buttons should appear, but they don’t do anything.
7.2. Import your new shaders
Near the top of
main.js you will need to import the new shader files you created with your new effects. Copy and paste
import fshader2 from "./fs_split.js";
and use the new names for your shader files. Give each one a new name (
fshader4 are fine for this week, we’re just learning).
main.js modify the
init(image) function to add two more
addEffect(…) lines that connect your button id names to shader import variables.
If all those steps are done correctly, your new effects should apply when you click their appropriate buttons. If something isn’t working, check the debugger console, or the
TODO points in the code.
Below are two possible effects using somewhat advanced filters, but still within a single shader. Feel free to experiment.
8. Closing the Web Server
When you are finished with a session, be sure to close your
ngrok is likely running in the foreground and can be stopped using
You can stop all your
python3 jobs using
pkill -u adanner python3
adanner with your username.
9. Summary of Requirements
Your project will be graded on the following components:
A working split function plus one additional image processing effect that was not an effect you implement in the CS35 pic filter lab.
Easy to read shaders.
Answer to concept questions in the
A small percentage of your grade will be based on style, creativity, and your new effect. A very simple or a snarky make blank effect meets the letter of the requirements, but not the spirit. Have fun and try something a little more elaborate.
You will not be graded on the lab survey questions in the
Once you have edited the files, you should publish your changes using the following steps:
$ git add <files you changed>
The git add step adds modified files to be part of the next commit to the github server.
$ git commit -m "completed lab1"
The git commit step makes a record of the recently added changes. The
-m "completed lab1" part is a descriptive message describing what are the primary changes in this commit. Making a commit allows you to review or undo changes easily in the future, if needed.
$ git push
git push command sends your committed changes to the github server. If you do not run git push before the submission deadline, I will not see your changes, even if you have finished coding your solution in your local directory.
If you make changes to files after your push and want to share these changes, repeat the
add, commit, push loop again to update the github server.
If you want to commit changes to files that have already been committed to git once, you can combine the add and commit steps using
$ git commit -am "bug fix/updates"
-a flag will automatically add files that have been previously committed. It will not add new files. When in doubt, use
git status, and please do not use
git add * ./
To recap, the git commit cycle is
git add, git commit, git push. Don’t forget to git push when you have completed an assignment.