ISE-311 Lab Facilities (Spring 2010)

Introduction

We have set up limited lab facilities for use by registered class students during the semester. The lab's machines are accessible only remotely and via a secure shell. Details are provided below. Note that the procedures we describe here will be updated and adjusted as needed throughout the semester.

There are restrictions on the use of the Lab facilities:

The lab facilities are limited. Start your homework early! Don't wait until the last minute. The more people compile code all at once, the slower the lab facilities will be. Vary your time of work: some people prefer day-time work, some like to work late at night, some over the weekends. Check who else is on your system (by running /usr/bin/w) and what the load is (using /usr/bin/uptime).

Getting Initial Access

To be able to do your homeworks, you need to do the following three things. They can be done independently.
  1. First, you'll need to ensure that you have physical access and login access to somewhere on campus, where you can sit down and do you work. You can use the Grad CS Labs, SINC sites, a personal workstation or laptop, etc.
  2. Second, you must subscribe to class mailing list. A lot of useful information will be provided via the mailing list. Through the list, we can distribute information much more quickly to the class students than during the weekly lectures.

    Note: I strongly encourage you to subscribe with your Stony Brook CS account. This will ensure that mail will arrive to you more quickly, and that you are less likely to run out of your mail quota than if you were using some other ISP's mail service. If I post an important message to the class list, and you don't get it because your mail quota has exceeded, I may get a bounce back saying so, but I will have no way to communicate back with you. It is therefore your responsibility to ensure that you can always read the class mailing list and to stay current with what I post there.

    Finally, please do not set your list subscription mode to Digest because digests can take several days to get emailed to you.

  3. Third, you need remote login access to the OSLAB, the lab that includes the machines on which you will program and test your programs. Note, you will not get physical access to this lab; you can only login to it remotely via SSH. Getting an OSLAB account takes several days, so start early. To apply for a OSLAB account, first register for the course, then use the ISE-311 Student OSLAB Account Request Form. Once you applied for this account, please wait 1-2 days for the account to get created (you will be informed via email, so make sure to put down a valid active email address you use).

    Please note that you will get an OSLAB account that will allow you to log into several machines and possibly also a UG lab account. Do not send email to the TA or the instructor from these machines, nor should you expect to get email sent to you on these machines; they are not configured for email processing. Please do all your email communication through whatever email means and email ID you were using outside these labs. It is particularly important that you follow these instructions to ensure that you could communicate with the teaching staff promptly.

  4. Lastly, if you plan to work remotely (off campus), then you'll need access through our OSLAB firewall. After you get your OSLAB account activated (you'll get an email confirmation), then visit this secure Web site to enable remote SSH access through the OSLAB firewall.

Secure Shell (SSH) Access to OSLAB

Firewalled SSH Access to OSLAB

The OSLAB lab housing the VMware machines is protected by a firewall. SSH access is allowed only from known IP addresses that have a reverse-resolved name in DNS. Access from inside campus is automatically allowed. If you try to ssh for the first time to any OSLAB machine (esp. from off campus) and you cannot get in, then you must enable your access one time. To do so, first get your OSLAB account activated, and then click here.

Note that if you use DHCP at the source (say, a DSL line at home), then your IP address may change over time. In that case, you will have to re-visit the above link periodically to re-enable your access for your new IP address Similarly, if you tend to ssh from multiple locations (home, work, etc.), then please revisit the above URL as many times as needed.

Be aware, however, that some times, the reason you cannot ssh into the OSLAB is due to a bad network connection somewhere outside campus, or outside the CS department: those situations are beyond my control. You are advised to start working on your homework early, and to ensure that you will always have a place to work on your assignments, even if much of the campus network is down (e.g., use the UG Lab).

I apologize in advance for this inconvenience. It was made necessary due to numerous break-in attempts daily through my lab SSH servers.

X11 Tunneling

Whatever SSH client you use, you should ensure that X11 tunneling is enabled; X11 Windows clients often require you to configure them specifically to enable X11 tunneling. Check the documentation for your SSH client for help; in OpenSSH, for example, the -X option enables X11 tunneling.

Trusted X11 Forwarding

Reliable X11 tunneling is required for VMware or any X11 client program to work properly. On several modern systems (e.g., Red Hat's Fedora Core 3, and others), the SSH client requires you to turn on "Secure X11 Forwarding" before you can reliably tunnel back an X11 connection. If you don't enable this secure forwarding, your X11 client may fail will all kinds of strange errors such as: The program 'vmware' received an X Window System error. This probably reflects a bug in the program. The error was 'BadAtom (invalid Atom parameter)'. If you get this error, then restart the ssh client with the '-Y' flag. See the man page for ssh(1) for more details.

BTW, sometimes, to help minimize network traffic over the SSH tunnel (especially. if you're working from home over a slow ISP), you can use use "ssh -C". The "C" is for "Compress" -- it tends to improve performance when you tunnel a X11 session (b/c the X11 protocol is a network hog :-)

Using Your OSLAB VMware Account

After you get your VMware account created, first try to log into it. You will need an SSH-compliant secure shell client tool, as well as run on a system running the X11 windowing server. For that, you may use the Graduate Lab or any other location where you can have SSH+X11.

Note that command instructions below are written assuming you use a shell like Bourne Shell (/bin/sh), KSH, or BASH. If you're a user of CSH or another shell, apply the appropriate commands in your shell of choice.

We have a pool of several VMware host machines, on which you will run a VMware guest OS. The general name for accessing the pool in round-robin fashion is vmpool.oslab.cs.sunysb.edu. (Our "pool" currently includes 4 servers, with 4-64GB RAM and 4-16 CPUs/cores each.) Those are the machines that run the VMware software. The actual Linux version you will run is called the Guest OS and will be started on top of VMware.

Before you login to the class machines, set may have to set your DISPLAY variable:

DISPLAY=:0.0 export DISPLAY This assumes that you are running X11 on the local host from which your logging in (most of you will not be). If you are tunneling via SSH, then you do not need to change your DISPLAY variable: in fact, changing it will prevent you from starting the VMware program.

Next, to login to one of the VMware host machines use the "vmpool.oslab.cs.sunysb.edu" name, which will pick one of the pool machines at random (thus helping to distribute the load among all class students):

ssh -Y user@vmpool.oslab.cs.sunysb.edu Where user is your username for the account that was given to you on the VMware/OS server (not your UG-Lab account name, which may be different). If you are using the same username, you can omit the "user@" part. The initial password you should use is the password you requested when you filled out the ISE-506 Student VMware Account Request Form.

SSH access is required and restricted by the lab's firewall: you can only come in from within the university, the CS department, and select addresses which I've authorized (e.g., subsets of optonline.net, etc.). If you cannot ssh into this host, it is likely that you are trying to connect from another host outside the university. To enable ssh access, you need to visit this secure Web site to enable remote SSH access through the OSLAB firewall. Note that you can always ssh first into one of the campus hosts, and from there ssh into the OSLAB hosts.

Once you log into oslab, you can start VMware, configure a new virtual machine, and install it. This could take 1-2 hours initially, if all goes well. One your VM is setup and initialized, you can start it (power it on). If you're installing Linux, then when the boot has finished, you will see a regular Linux Console login, and you will have a Linux system exclusively for your use (i.e., you can crash it, reboot using CTL-ALT-DEL, etc.). The name that is assigned to your guest OS is going to start with "dhcp".

You may now log into this machine with the User-ID and password you setup when you installed your VM.

WARNING: Do not share the root VMware password with anyone else outside class!!! In addition, do not use your vmware root privileges for anything other than to perform your homework assignments for this class. Any deviation or abuse of your privileges will be result in your account disabled, a grade of "F" for the class, and a referral to the Academic Judiciary Committee.

Troubleshooting VMware when it no longer starts

If VMware worked for you before, but now you get an error when trying to start it, it could be because of leftover VMware processes running on one or more of the VM pool of servers. This happens most often if your network connection to the VM pool had been severed unexpectedly, or if you didn't shutdown vmware cleanly (e.g., don't click the "X" button on your VMware window, but use the proper way through its menu options File->Quit or File->Exit). The error you get in this case is often something like "vmware already running" or "cannot lock virtual machine". If you are absolutely certain that you no longer have a running VM, then do the following:

  1. Login to each of the VM pool machines in order (o3.oslab.cs.sunysb.edu, as well as o4, o5, and o6), and check if you have any runaway processes: $ ps -ef | grep `whoami` If you see any process named "vmware" or "vmware-*" (e.g., vmware-vmx), then kill it by PID using "kill PID" where PID is the process ID of the one you want to terminate. Wait a few seconds, then check again if there are any leftover vmware processes: if so, kill them with "kill -9 PID."
  2. Once you've determined that you have no more leftover vmware processes, you need to remove leftover lock files/directories, if any. This needs only be done once from any of the vmpool machines: $ /bin/rm -fr ~/vmware/*/*lck
After these two steps, you should be able to restart vmware.

Using your own personal Linux/Windows/etc. machines

We strongly recommend that you do all of your homework assignments on the OSLAB VMware machines. That way you can benefit from our experience using these machines, as well as the experiences of other students in the class.

Nevertheless, a number of people experienced with other systems sometimes choose to work on their assignments elsewhere: at home, at work, in a dorm, on a personal laptop, etc. That is OK. However, it is solely your responsibility to ensure that VM and your code is working and properly committed on the OSLAB VMware machines! If you choose to develop your code outside the OSLAB, be sure to give yourself enough time to migrate it into the VMware machines, test it there, debug it as needed, etc. Don't assume that just because it worked for you outside the lab, that it'll automatically work in the lab: there could be many reasons why your code may not work in other settings. We will not allow people to bring laptops or other machines to demo their code. We will not accept excuses such as "but it worked for me at home" or "it really works --- I just need to copy it over."

If you do choose to work elsewhere on your assignments, we recommend that you duplicate the same exact environment we have on the VMware machines (e.g., CentOS 5.4 or whatever we use currently). We will try to help you to set up your personal machine appropriately, but please realize that our time and resources are limited to official class matters. Unfortunately, we do not have the time to help each student in the class to set up a separate, personal work environment, especially on esoteric machines. That's why we worked hard to prepare a standard work environment using VMware for the entire class.

Compiling Kernels and Other Sources

I suggest you compile inside your VMware machine (oslab), in the standard directory /usr/src. The following annotated example shows you step by step the commands needed to build a Linux kernel:
  1. Login to oslab:
    ssh user@vmpool.oslab.cs.sunysb.edu
    
  2. Start Vmware
    vmware &
    
    Note: from now on, all your commands should be executed inside your own virtual machine, the one you just rebooted.

  3. Boot your Vmware (Linux) image, login as yourself, and then cd to your work directory:
    cd /usr/src
    
  4. Retrieve the Linux kernel sources via FTP. Use one of these two commands
    wget ftp://ftp.fsl.cs.sunysb.edu/pub/linux/kernel/v2.6/linux-2.6.32.6.tar.bz2
    
    or
    ncftpget ftp://ftp.fsl.cs.sunysb.edu/pub/linux/kernel/v2.6/linux-2.6.32.6.tar.bz2
    
    Note that the first assignment may ask you to use Linux version 2.6.32.6. Subsequent assignments may be required to use a different version of the kernel. In that case, replace "2.6.32.6" in the instructions the follow with the newer version of the kernel.
  5. Unpack tarball into ./linux-X.Y.Z.nn (in /usr/src):
    tar jxf linux-2.6.32.6.tar.bz2
    
  6. To save space, remove the tarball (you can always re-get it)
    rm linux-2.6.32.6.tar.bz2
    
  7. Change your working directory into kernel source directory
    cd linux-2.6.32.6
    
  8. Configure a kernel for running on VMware (you can also use "make config" for line-by-line configuration or "make xconfig" for an X11 graphical configuration).
    make menuconfig
    

    An easier alternative to configuring a kernel is to use one of my own kernel configuration files, designed for VMware:

    cd linux-2.6.32.6
    wget http://www.cs.sunysb.edu/~ezk/ise311-s10/vmware.config
    mv kernel.config .config
    make oldconfig
    
    Note that if you choose to use my pre-configured kernel configuration file, you still have to know how to configure Linux kernels (and you'll be tested on it).
  9. To build the kernel and modules (also can take time)
    make
    
At this point, your kernel and modules are built, and you're ready to install them.

Note: in the following instructions, commands which appear in red are executed as root. Then run these commands:

  1. Become root using the root password given to you in class
    /bin/su -
    
  2. Install modules:
    
    make modules_install
    
    
  3. Install the main kernel image:
    
    make install
    
    
  4. Normally, "make install" will update your kernel boot file which lists all of the kernels you can boot. If (and only if) it doesn't, then you'll have to update it yourself. Using a text editor such as emacs (or vi, pine, jed), add a kernel boot entry to the end of the boot loader configuration file grub.conf:
    
    vi /etc/grub.conf
    
    
    The entry you can add has the following text:
    
    # to boot from the first IDE drive, first partition
    title test kernel (2.6.32.6 ide)
    	root (hd0,0)
    	kernel /boot/vmlinuz-2.6.32.6 ro root=/dev/hda1
    # to boot from the first SCSI drive, first partition
    title test kernel (2.6.32.6 scsi)
    	root (hd0,0)
    	kernel /boot/vmlinuz-2.6.32.6 ro root=/dev/sda1
    
    
    Be careful not to change anything else in grub.conf, or you could render your system unbootable and then you'll need to reinitialize your vmware. In particular, leave behind the original entry for the default Red Hat kernel. When you're done adding the new kernel entry to grub.conf, save the file, and exit the editor.
  5. Finally, reboot your system
    
    reboot
    
    
    When the system will boot, you will see a GRUB menu that now lists two possible kernels to boot. The first (default) one is the one that came with the system and is always good to boot into as a backup solution. The second entry should be the one that you just installed. Use the arrow keys to select the second one (within 10 seconds) then hit ENTER to select it. Your system should now boot your new kernel. When it comes up, login as yourself to ensure that everything appears to be in working order.
All of the procedures described in this section are just to get a new, generic, unmodified (i.e., vanilla) kernel built, installed, and booted. Now that you've done that, you can move on to trying to build sources for the homework assignment, either as independent modules or as modified kernel sources, and try your changes.
Last Updated: 2010-01-26