CS 43 — Lab 4: Designing a Jukebox Protocol

Due: Thursday, April 7, 11:59 PM ET (All code must be pushed by then, despite demos coming after.)

Due: Friday, April 8, 11:59 PM ET (All code must be pushed by then, despite demos coming after.)

See EdSTEM for information about a bug in the starter code!

1. Overview

For this lab, you’ll design and implement your very own music streaming protocol. Since the protocol is your own design, you’ll need to develop both the client and the server. The protocol you design will support a few basic operations that the user can request from a client, including retrieving a list of songs, getting information about a song, playing a song, and stopping playback.

1.1. Goals

  • Design your own application-layer protocol.

  • Implement both the client and server sides of communication based on your protocol design.

  • Develop a persistent server to connect with multiple interactive clients using event-based concurrency.

  • Apply the producer-consumer threading model to interact with the user while transferring data in the background.

1.2. Handy References:

1.3. Lab Recordings

Week 1, Board Photo

Week 2, Notes

Week 3

2. Requirements

You must use C to implement your server, and it must use select (rather than threading) to support multiple concurrent clients. Your server will receive two command line arguments: a port number on which to listen for incoming connections and the name of a directory with music files. For example:

$ ./server [port] /home/kwebb/music/
Found 15 songs.

You may use any language you wish for the client, and you may use threads in the client. I strongly recommend Python for the client, as it makes playing audio much simpler. The client expects to receive two command line arguments: the host name (or IP address) of the server and the port that the server is listening on.

$ ./client.py [hostname].cs.swarthmore.edu [port]
>>

You need to fill in the [hostname] and [port] fields above with the host name and port your server is running on. If you want to test with a rate-limited server, be sure to use a port in the range 5001 - 5099 on moltres.cs.swarthmore.edu. If that’s what you want to do, you must ssh to moltres before running the server.

Your client should be interactive, with users typing any of the following commands to request jukebox behavior at any time:

  • list: retrieve a list of the songs available at the server.

  • info <song number>: retrieve text information about the specified song number.

  • play <song number>: retrieve the bytes of the specified song file (in MP3 format) for the client to play the song.

  • stop: end the song data file transfer (if data is being transferred) and stop playing the current song (if one is playing).

  • exit: disconnect the client.

The client should not cache data. In other words, every time a user enters a command, the client needs to send a request to the server for the corresponding data.

The protocol you design (and your implementation of it) must allow list and info messages to be interleaved with song data. That is, while the server is sending data in response to a play command, the user should still be able to request the list of songs or information about songs without needing to wait for the entire song file to transfer first.

2.1. Protocol Design & Server State

Thus far, we’ve seen a few different types of application-layer protocols that follow the client-server architecture. Having discussed various protocol design trade-offs, consider at least the following when designing your protocol:

  • Header information: what information must be in the header of each message to ensure correctness?

  • Field format and size: for each piece of information that you exchange in a message, how large will it be and how will the receiver know the size? How will it be formatted (e.g., text vs. binary, byte order, etc.)

  • Message delimiters: how will each side of the communication recognize where one message ends and the next one begins?

  • Per-client state: what does the server need to maintain to keep track of the state of a client? What variables does it need to store and what do they mean?

Common Gotchas

Students often neglect to consider these cases:

  • What happens when a user enters stop at a client? If it immediately stops playing any music data it has buffered, what should happen if new data continues to arrive afterward?

  • Does your protocol allow for interleaving of messages? For example, if the user asks to play a large song file and then enters list while the file is still transferring, how does your protocol reply with the list response without the user needing to wait for the entire file transfer to complete?

  • When keeping track of the state of a client at the server, account for the case when you call send and fewer than len bytes were sent (that is, the return value of send indicated that some, but not all of the bytes got sent). The next time you send to the same client, you need to resume sending the old message where you left off rather than generating a new one. What sort of variables do you need to keep track of this case?

I would like to have a brief (10-minute) protocol specification and progress meeting with each group on or before labs start on March 25. Please sign up for a time. For this meeting, you should come prepared to discuss the details of your protocol and the per-client state you intend to keep at the server.

2.2. Server

2.2.1. Workflow

  1. Start up and read .mp3 and .info file data into in-memory storage (e.g., a global array). You may also want to build a string to handle list messages, since that won’t change.

  2. Begin listening for incoming client connections.

  3. Infinitely execute the main event loop, which uses the select function to decide:

    1. Is there a new client that is connected for us to accept? If so, accept the new socket and initialize the state associated with that new client. (This is true when select says that the server socket is ready for reading.)

    2. Have any clients sent data to the server? If so, for each client, receive the data and update the client’s associated state to reflect what they’ve just requested.

    3. Do any clients that the server would like to send a message to (i.e., the state of the client indicates that it has messages pending) have space available in the outgoing socket buffer? If so, for each client, look at the state of the client and send the appropriate data to it.

When sending song data, it’s helpful to send relatively large chunks at a time (e.g., 16 or 32 kilobytes). In my solution, I initially sent 4-kilobyte chunks, and song playback stuttered at the start. Switching to 16-kilobyte chunks seems to work much better.

2.2.2. Expectations

  • After it starts up and begins serving clients, your server should never block on any call other than select. Blocking on a send, recv, or accept is a bug!

  • If a client disconnects from your server, for any reason, your server should not crash. Instead, it should reset the disconnected client’s such that it will ignore it until some other client comes along that happens to use the same socket descriptor number. You might detect a client leaving in two ways:

    1. When a client disconnects, if you’re blocked on a call to select, the select call will return and the disconnected client’s socket will show up as readable. Calling recv will return 0 to indicate "end-of-file". If you see a return value of 0 from a client socket, that client is gone.

    2. When you call send on a client socket, if the client has disconnected, your call to send will fail and generate a SIGPIPE signal, which by default will terminate your entire server process. If you pass the MSG_NOSIGNAL flag to send, it will not deliver the signal and instead return -1. If you see a return value of -1 from send, the client is gone.

  • When using a language without garbage collection (e.g., your server in C), you should generate no valgrind warnings. Because the server is still active when you hit CTRL-C, it’s fine for memory to be still reachable, but none should be lost.

2.3. Client

2.3.1. Workflow

When you start the client, it will create two additional threads (for a total of three threads):

  • The main thread, which reads user input commands and, when the user enters a command, sends the corresponding request to the server. Its sequence is:

    1. Create the other two threads.

    2. Enter an infinite loop that waits for user input. When the user inputs a command, build a request and send it to the server.

  • The receiving thread, which always tries to receive and process data from the server. Its sequence is (in an infinite loop):

    1. Receive a message header.

    2. Receive a message body.

    3. If the message contains a text response (e.g., in response to list or info), print it.

    4. If the message contains song data, add the new song data to any other song data you’ve already buffered and inform the playing thread that new data is available. Because the song data is shared with the playing thread, you need to protect access to it so that both threads don’t attempt to access it at the same time.

    5. If your protocol specifies other message types, handle those as needed. (You may find it helpful to get stop messages too.)

  • The playing thread, which waits for song data and plays it when available. Its sequence is (in an infinite loop):

    1. Wait for song data to become available. When data arrives, process it and play it.

The receiving thread and playing thread have a producer/consumer relationship. As the receiving thread receives song data, it produces data by appending it to the outstanding data that needs to be played. The playing thread consumes the data by working through from start to end. This relationship means you need to guard against two potential problems using thread synchronization:

  1. Only one of the two threads should access the song data at a time, so you need mutual exclusion.

  2. The player thread needs to worry about data underflow. That is, it shouldn’t attempt to play song data when no song data is available. Thus, you want it to wait on a condition variable that will indicate that new data has arrived. The receiving thread will need to signal the condition variable to wake up the sleeping playing thread every time it adds new data.

In Python, a condition variable implicitly contains a mutex lock. You should create one condition variable and share it with both threads. Threads can call .acquire() and .release() on it to lock or unlock the mutex, respectively.

While holding the lock, a thread can call .wait() on the condition variable to wait for some future event to occur (e.g., data arriving). Calling .wait() implicitly gives up the lock and automatically reacquires the lock upon returning.

While holding the lock, a thread can call .notify() on the condition variable to unblock a thread that is waiting on it (e.g., to let it know that data has arrived). If there is a waiting thread, it will wake up from its call to .wait() as soon as the lock becomes free. If no threads are waiting, then nothing happens.

2.3.2. Expectations

  • A user should be able to request that a song start playing and then immediately (while the song data is still transferring) be able to request text via list or info without having to wait for the file transfer to complete.

2.4. Assumptions / Simplifications

  • To simplify the file I/O, it’s fine for the server to keep its data, including the audio files, in memory, but the client should not store data that it isn’t actively using.

  • You may assume that the client has infinite buffering capacity for song data. That is, unlike "real" streaming protocols, you don’t have to keep track of how much data the client has processed before sending them more data. Once you start sending song data, just keep sending more until you reach the end (unless the client requests something else in the meantime).

  • One of the parameters to your server is a path to a directory that contains audio files and their corresponding information. Within this directory, you may assume that any file ending in .mp3 is an mp3 audio file. For each mp3 file, there will be a corresponding information file that is identical to the file name with .info tacked on to the end. For example, if there were a file in the directory named song1.mp3, there will also be a file named song1.mp3.info containing human-readable plain text information about that song. This info file is what should be supplied when the client issues an info command. I’ve made my music directory publicly available at /home/kwebb/music/, and I’ve provided code to read the names of these files. Feel free to use that or your own mp3 files for testing, but please do not submit audio files to GitHub!

2.5. Checkpoints

By the end of the first week, you should have written down:

  1. A specification for your protocol, including header formats, field sizes, and anything else someone might need to know to implement your protocol.

  2. A list of the state variables you intend to keep at the server and what each variable means.

These documents don’t have to be super long or formatted like RFCs, but they should provide enough detail for you to refer to as you work on your implementation. They also aren’t set in stone — you may have to change the design slightly as you encounter implementation challenges, but my expectation is that that they won’t change too drastically.

By the end of the second week, you should:

  • Be able to accept clients at the server.

  • Be able to exchange text (list and info) requests and replies between multiple concurrent clients and the server.

3. Examples and Testing

See the recordings for a demo.

3.1. Slowing the Server

To test message interleaving (handling text messages for a client that is also receiving song data), it helps to have a server that’s sending slower than our typical gigabit network. I’ve set up the machine moltres.cs.swarthmore.edu to artificially slow done its sending rate when sending from ports in the range 5001 - 5099.

Execute your server on moltres and bind to your favorite number within that port range. Try playing a song and then immediately making a list or info request. How long do you have to wait? If you’re properly interleaving messages, the delay should be short!

Sending from those ports, you’ll get a maximum throughput of approximately 256 kilobytes per second, so it will take about 20 seconds to fully transfer a 5-megabyte file. The files in my public music directory range from 2.5 MB to 13 MB, so that should give you plenty of time to test a play command followed by list or info.

4. Tips & FAQ

  • Use send and recv in as few places as possible. Each call to these functions significantly increases the complexity of debugging and reasoning about your server. It’s possible to construct the server such that it only calls send and recv in one place each. Doing so is a solid design goal.

  • Never call send, recv, or accept on a socket unless select has told you that it’s safe to do so. Otherwise, you might block (and potentially deadlock!).

  • It is critically important that you check the return value of any system call you make. If a function is failing, you need to know why.

  • If your server is crashing due to the SIGPIPE signal, pass the MSG_NOSIGNAL flag to your send calls. With that flag, if send returns -1, the client has disconnected, and you shouldn’t attempt to send to or receive from it further. You can test this by making a play command at a client and then terminating the client with CTRL-C shortly afterwards.

  • You can set your sockets to non-blocking mode for debugging by calling the provided set_non_blocking(int sock) in your starter code. This is not required (you should never block regardless because you should never try to send, recv, or accept unless select tells you it’s ok), but it will prevent deadlocking during your testing. Check their return values and the errno variable for EAGAIN/EWOULDBLOCK, which indicates that you made a system call that you shouldn’t have. That’s send / recv / accept's way of telling you "I would have blocked on this call had this socket not been set for non-blocking mode.

  • If you want your client to begin playing a new song, you need to create a new MadFile object. This tells mad (our mp3 decoder library) to interpret the next bytes as the beginning of a new file (which includes some metadata) rather than the middle of the previously playing file.

  • Wireshark will not help you for this lab, since you’re designing the protocol this time. Wireshark knows nothing about how to decode your protocol!

5. Other Reference Material

5.1. Condition Variables

Some general information about condition variables is available in the Dive into Systems textbook.

5.2. Event-based programming and select

In an event-based concurrency model, a server uses a single thread for all client requests. Essentially, it uses a function like select to determine which client sockets are safe to receive from or send to. Then, it cycles through all of the ready sockets and performs just one operation for each client. In this way, it makes a little bit of progress for each client and then repeats the whole process. Conceptually, you can think of this like OS process scheduling on a CPU — the OS doesn’t just let one process run to completely, but instead it context switches each process across the CPU for short periods of time, allowing each to make progress.

See the man pages for select and select_tut.

The input parameters for select are:

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);
  1. nfds: The first argument to the select function is (the largest numerical socket descriptor value in any of the subsequent fd_set parameters) plus one. For example, if you’re populating your fd_set variables with socket descriptors 7, 9, and 10, your first argument to select should be 11 (10 + 1).

  2. fd_set: select takes input variables that, by convention, are named rfds and wfds of type fd_set. These are sets of file or socket descriptors (fd stands for file descriptor). As far as the OS is concerned file and socket descriptors are equivalent. You can manipulate fd_set sets in a few ways:

    1. void FD_ZERO(fd_set *set);: removes all descriptors from a set.

    2. void FD_SET(int fd, fd_set set);: add the specified descriptor to a set. For example, FD_SET(serv_sock, &rfds) puts the server socket (the socket the server calls bind and listen on) into the read set. You should *always put the server socket in the rfds set — when the server socket is ready for reading, it means there’s a new client to accept.

  3. For this lab, you can safely set the exceptfds and timeout to NULL — we don’t need them.

When select returns, it will remove any sockets that are not ready from the fd_set parameters, leaving only those that are ready. After select returns, you can check whether a descriptor remains in a set using int FD_ISSET(int fd, fd_set *set);. It will return non-zero (true) if the descriptor is still in the set or zero (false) otherwise.

If select leaves a socket descriptor in a fd_set after it returns, it means that you can safely (without blocking) recv (if in rfds) or send (if in wfds) on that socket once, but not more than once. After performing one recv or send, you must get approval from select again before attempting to send or receive on the same socket again. Otherwise, the second operation might block.

If a client closes a connection while your server is waiting for select to return, select with keep that client’s socket in the set of file descriptors that are available for reading (rfds). Then, when your server goes to call recv on that socket, it will get a return value of 0, which as we saw in lab 1, indicates that the connection is closed and no more data is coming — we’ve reached the end-of-file (EOF).

Thus, as I’ve been harping on all semester, you should always check the return value of your system calls to check for these types of conditions so that your server can detect and account for disconnecting clients.

5.3. Byte Ordering

In Python, all the packing and unpacking with the struct module that you used in lab3 still applies here.

In C, to pack data, you can create a buffer like you normally would: char data[] or char * data = malloc(…​).

  • To pack or unpack a single byte you can just index into the array using data[idx].

  • To pack or unpack more than one byte the byte-ordering functions are as follows:

    "Host to Network, short"
    htons() - convert a short (2-byte int) from host byte order to network byte order
    
    "Network to Host, short"
    ntohs() - convert a short (2-byte int) from network byte order to host byte order
    
    "Host to Network, long"
    htonl() - convert a long (4-byte int) from host byte order to network byte order
    
    "Network to Host, long"
    ntohl() - convert a long (4-byte int) from network byte order to host byte order
  • For example, when receiving data, if you know that you have a short contained in byte positions 3 and 4 of a buffer, you can call ntohs on it as:

    short result = ntohs(*((short *) &buf[3]));

    This line says: take the address of position 3 in buf and cast it to be a short pointer. Then, dereference the pointer to the short data (positions 3 and 4, since short is a two-byte type) to get a short and then call ntohs on it to swap the byte order to host format so that your program can use it.

  • When preparing a message for sending, you can call htonX on a variable to convert it to network byte order and then use memcpy to pack it into a buffer.

6. Submitting

Please remove any excessive debugging output prior to submitting.

To submit your code, commit your changes locally using git add and git commit. Then run git push while in your lab directory.