Introduction

There are restrictions on the use of the Lab facilities:

• Only registered users may use it.
• Auditors cannot use the lab.
• Access to the lab requires SSH. If you are running on a host that does not have SSH installed (for example on your Windows XP laptop from your dorm), then download a commercial one from DoIT, and then install it; alternatively, you can get a freely available version called Putty.
• Lab use is for class work only and only for this class. DON'T USE THIS ACCOUNT FOR YOUR OTHER CLASSWORK OF ANY OTHER CLASS. Students caught using this account for other classes will have their accounts turned off and all files related to other classes deleted.
• People caught "hacking", cheating, or abusing the resources will be denied access to the facilities and be handled in accordance to the university's policies on academic dishonesty and computer use.

Getting Initial Access

1. First, you'll need to ensure that you have physical access and login access to somewhere on campus, where you can sit down and do you work. You can use the Grad CS Labs, SINC sites, a personal workstation or laptop, etc.
2. Second, you must subscribe to class piazza forum (if you're not already subscribed by me). A lot of useful information will be provided via piazza. Through the forum, we can distribute information much more quickly to the class students than during the weekly lectures.

   Note: I strongly encourage you to subscribe with your Stony Brook CS account. This will ensure that mail will arrive to you more quickly, and that you are less likely to run out of your mail quota than if you were using some other ISP's mail service. If I post an important message to the class list, and you don't get it because your mail quota has exceeded, I may get a bounce back saying so, but I will have no way to communicate back with you. It is therefore your responsibility to ensure that you can always read the class mailing list and to stay current with what I post there.

3. Third, you need an account on the CS network. If you are a registered student in this class, your account is already active. You don't need to apply specially for it. (This is a simplification/improvement over past methods.)

4. Fourth, you need remote login access to the OSLAB lab that includes the machines on which you will program and test your programs. Note, you will not get physical access to this lab. You will need two forms of access: SSH to your personal virtual machine, and a Remote Desktop (rdesktop) method to the VMware ESX Console. Both are described below in greater detail

   Please note that you will get a personal VM account that will allow you to log into several machines. Do not send email to the TA or the instructor from these machines, nor should you expect to get email sent to you on these machines; they are not configured for email processing. Please do all your email communication through whatever email means and email ID you were using outside these labs (e.g., CS's Gmail email system). It is particularly important that you follow these instructions to ensure that you could communicate with the teaching staff promptly.

Remote Desktop (RDP) Access to your Virtual Machines

Note, find out which ESX server you were assigned to. So you can login to that server for your VM. Consult this assignment list vm-assignments.txt.

Using Your Personal VM

Once you login, you can sudo to gain root privileges, using the same passwd for user 'ubuntu':

You can only login to the vlXXX machine using user 'ubuntu', not your AD password. The default 'ubuntu' user password for your VM will be given to you in class. Please do not share it with anyone else. And please change it as soon as you login, to prevent others from logging into your VM inadvertently.

WARNING: Do not share the 'ubuntu' VMware password with anyone else outside class!!! In addition, do not use your vmware privileges for anything other than to perform your homework assignments for this class. Any deviation or abuse of your privileges will be result in your account disabled, a grade of "F" for the class, and a referral to the right authorities as per the academic dishonesty policy.

Troubleshooting when your VM no longer starts

Using your own personal Linux machines

Nevertheless, a number of people experienced with Linux sometimes choose to work on their assignments elsewhere: at home, at work, in a dorm, on a personal laptop, etc. That is OK. However, it is solely your responsibility to ensure that your code is working and properly committed on the OSLAB VMware machines! If you choose to develop your code outside the OSLAB, be sure to give yourself enough time to migrate it into the VMware machines, test it there, debug it as needed, commit it to CVS, check out a test copy and test it, etc. Don't assume that just because it worked for you outside the lab, that it'll automatically work in the lab: there could be many reasons why your code may not work in other settings. We will not allow people to bring laptops or other machines to demo their code. We will not accept excuses such as "but it worked for me at home" or "it really works --- I just need to copy it over."

If you do choose to work elsewhere on your assignments, we recommend that you duplicate the same exact environment we have on the VMware machines (Red Hat Linux 9 with all of the latest patches, configured properly). We will try to help you to set up your personal machine appropriately, but please realize that our time and resources are limited to official class matters. Unfortunately, we do not have the time to help each student in the class to set up a separate, personal work environment, especially on esoteric machines. That's why we worked hard to prepare a standard work environment using VMware for the entire class.

Compiling Kernels and Other Sources

Note also that the VM you get is minimally installed. It doesn't have even compiler tools like gcc, make, etc. You will need to use Ubuntu's "apt-get install" command to install the packages you will need. You'll find out what's missing when you try to build the kernel.

The following annotated example shows you step by step the commands needed to build a Linux kernel:

1. SSH to your VM:

   ssh -p 130 ubuntu@myhost.cs.stonybrook.edu
   

   where myhost is the DHCP hostname of your VM (e.g., "vl321" as shown above). You can only login as 'ubuntu' to the dNNN machine, with the password that will be given in class; don't try to use your AD password.

   Note: from now on, all your commands should be executed inside your own virtual machine, the one you just rebooted.

   cd /usr/src
   

2. Retrieve the Linux kernel sources via GIT. Each of you have been given a personal GIT repository for each homework assignment.

   git clone ssh://user@scm.cs.stonybrook.edu:130/scm/cse506git-s22/hw1-user
   

   where user (twice in the above command) is your AD username. You'll be prompted for your CS AD password.

   The initial checking out (or "cloning") of the git repository will take some time. Using your checked out repository afterward will be faster. Note that the first assignment must be done in Linux version 5.x (where "y" is whatever's the latest sub-release). The source code repository cloned for each student already uses a specific kernel version for you to use. Subsequent assignments may be required to use a different version of the kernel. In that case, replace "5.x" in the instructions the follow with the newer version of the kernel.

3. Change your working directory into kernel source directory

   cd hw1-user
   

4. Configure a kernel for running on VMware

   CAUTION: While I provide some detailed instructions below, your general goal is to build the smallest kernel you can, with as few features as possible, such that it does not failed to boot or produce any errors during the boot of the (CentOS or Ubuntu) Linux image. If you take off too many features, your system may not boot, or will boot with errors. Note that sometimes it is also important whether a feature is enabled as a loadable module or compiled statically into the kernel image.

   Use the command below to configure your kernel. You can also use "make config" for line-by-line configuration.

   make menuconfig
   

   • For a VMware kernel, be sure to include support for IDE disk/cdrom driver, floppy disk, Ext2 file system, Ext4 (plus JBD) file system, NFS client, networking (Packet Socket and Socket Filtering), Intel E1000 PCI Ethernet card, Loop block-device support, RAM-disk support (as well as "initrd"), and whatever else you think you may need. To make your life easier, turn on support for these features as statically linked code, not as modules.
   • Be sure to turn off CONFIG_MODVERSIONS (in Loadable Module support) as it could cause some problems when compiling loadable modules.
   • You may also need CONFIG_VMXNET3=y (VMware's paravirtualized network driver).
   • Additional useful things to turn on are CONFIG_EXPERIMENTAL (in Code Maturity), CONFIG_MAGIC_SYSRQ (in Kernel Hacking).
   • Other things you can safely turn off to make your kernel image smaller are support for SMP, PCMCIA, AGP, Kernel Automounter and NFS server (in File Systems), Sound cards, and USB. Use your own judgment.
   • Since there are so many features in the 5.x kernel you don't need for this class, you'll probably need to turn off more things than turn them on. By doing so, you will ensure that your kernel is small, compiles quickly, and runs even more quickly.
   • When you're done configuring your kernel, be sure to save your new configuration. If you made a mistake, you can always restart the configuration command. If you're interested in seeing your actual configuration, check the contents of the file ".config" inside the kernel source tree.
   • On some kernels/distros, you must also enable CONFIG_SYSFS_DEPRECATED_V2.

   Note: if you find that certain packages are missing on your VM, you can install them using the "yum" utility. For example, to fully run the menuconfig option, you may have to install the Ncurses development library with:

   yum install ncurses-devel
   

5. To build the kernel and modules (also can take time), run

   make
   

Note: in the following instructions, commands which appear in red are executed as root. Then run these commands:

1. Become root using the sudo password given to you in class

   sudo bash
   

2. Install modules:

   
   make modules_install
   
   

3. Install the main kernel image:

   
   make install
   
   

4. Normally, "make install" will update your kernel boot file which lists all of the kernels you can boot. If (and only if) it doesn't, then you'll have to update it yourself. Using a text editor such as emacs (or vi, pine, jed), add a kernel boot entry to the end of the boot loader configuration file grub.conf:

   
   emacs /etc/grub.conf # or grub2.conf
   
   

   The entry you can add has the following text:

   
   # to boot from the first IDE drive, first partition
   title test kernel (2.6.30.5 ide)
   	root (hd0,0)
   	kernel /boot/vmlinuz-2.6.30.5 ro root=/dev/hda1
   # to boot from the first SCSI drive, first partition
   title test kernel (2.6.30.5 scsi)
   	root (hd0,0)
   	kernel /boot/vmlinuz-2.6.30.5 ro root=/dev/sda1
   
   

   Be careful not to change anything else in grub.conf, or you could render your system unbootable and then you'll need to reinitialize your vmware. In particular, leave behind the original entry for the default Red Hat kernel. When you're done adding the new kernel entry to grub.conf, save the file, and exit the editor.
5. Finally, reboot your system

   
   reboot
   
   

   When the system will boot, you will see a GRUB menu that now lists two possible kernels to boot. The second (default) one is the one that came with the system and is always good to boot into as a backup solution. The first entry should be the one that you just installed. Use the arrow keys to select the first one, the one you just installed, (within 10 seconds) then hit ENTER to select it. Your system should now boot your new kernel. When it comes up, login as yourself to ensure that everything appears to be in working order.

GIT: Submitting, committing, and backing up your code

Do not wait until the last minute before submission time to figure out how to submit your homework assignment. Learn these procedures well ahead of time, experiment with them, and try and try again. You can submit your assignment as many times as you'd like, but we will consider the last set of files you've submitted as the ones you want us to grade. Don't be late because we won't accept late submission excuses such as "GIT didn't work for me."

Warning: protect your homework from accidental loss! All of the files you have in your VM directories are not being backed-up. That means that if you remove your files by mistake (happens a lot, often right before a due date), you will lose all of your data and won't have anything to submit. Using GIT you can make a copy of your files and more: record each version of your files as you change them. It is your responsibility to use GIT often. The graders will not accept excuses such as "I just removed all my files and can't submit my homework."

1. SSH to your VM:

   ssh -p 130 ubuntu@myhost.cs.stonybrook.edu
   

   where myhost is the DHCP hostname of your VM (e.g., "vl321" as shown above).

   cd /usr/src
   

2. Retrieve the Linux kernel sources via GIT. Each of you have been given a personal GIT repository for each homework assignment.

   git clone ssh://user@scm.cs.stonybrook.edu:130/scm/cse506git-s22/hw1-user
   

   where user (twice in the above command) is your AD username. You'll be prompted for your CS AD password.

   The initial checking out (or "cloning") of the git repository will take some time. Using your checked out repository afterward will be faster. Note again that this checkout procedure must be done only once: repeating it can confuse GIT into using recursive repositories, and your homework could be lost.

3. Next, you can modify any existing source file, then type "make" to compile your changes, install them, and test them.

4. If you add new files to the repository, you will have to tell GIT to commit them in the repository:

   git add newfile.c
   

   If you have other files to include in your submission, you should "git add" them as well. But, do not submit or add/commit binaries or other object files: those can be produced easily by running "make".

5. If you wish to create subdirectories under a certain GIT repository, you can do so as follows. For example, to add a subdirectory "test":

   mkdir test
   git add test
   

6. You can check your uncommitted changes using:

   git diff
   

7. You can list which files have changed using:

   git status
   

8. When you are ready to commit changes, you first have to update the GIT index of any changed file:

   git update-index changedfile.c
   

9. After updating the index of the file, you will have to commit it to your local repository:

   git commit
   

   Be sure to provide a suitably detailed commit message to explain what you did.

10. Note: at this point, all git did is commit your changes to the local repository only. These changes are NOT safe from VM corruption (and if you revert to an earlier snapshot, you will lose all your committed changes!). To ensure that your changes have made it to the remote (and save) copy on the "scm.cs.stonybrook.edu" machine, you have to "push" the changes:

    git push
    

11. You can also check what was committed using

    git log | more
    

Lastly, you can check out a second, temporary version of your own committed files and verify that they compile and run as you expect them to:

cd /tmp
git clone ...
cd hw1-user
git log | more
# configure/test kernel, etc.