OSI Lab1: Setup and hello world

Lab1 comprises two parts:

Part 1: Setup and tools
Part 2: Hello world in egos-2k+

The first part guides you to install the required tools and setup the lab environment. The second part serves as a warming-up to familiarize yourself with egos-2k+. There is a lot to read, but just tens of lines of code to write.

Quote from the Labs page, “if a lab is at first confusing even in its instructions, please don’t be discouraged; understanding the instructions is part of the work of the labs!”

Part 1: Setup and tools

We are going to use git, qemu, and Linux or MacOS for our labs.

Section 0: GitHub

If you don’t have a GitHub account, sign up for one here. You only need a Free plan for the labs.

Section 1: Prepare your host OS: Linux or MacOS

In this course, you will run egos-2k+ on qemu (a full-system emulator), which further runs on either Linux or MacOS.

If you’re using:

Windows 10/11: you can do our labs with the Windows Subsystem for Linux (WSL). If you have WSL, you’re good to go to section 2.
- Note: how to install WSL is here.
Linux: you’re good to go to section 2.
- Note: we’ve only tested labs on Ubuntu. But, in principle, you should be able to use other Linux distributions as well.
MacOS: you’re good to go to section 2.
- Note: you may face a permission problem when running labs for the first time. Please give permissions to all tools the labs use. See HowTo here. If you want to grant permissions without using GUI, see this post.

Section 2: Cloning OSI environment

We assume you know the basics of git. If not, please read what is git? in appendix.

We assume you have an ssh-key on your working machine that can git clone/push/pull your GitHub repos. If not, please read Teaching GitHub about your identity in appendix. If you don’t know, use $ ssh git@github.com to confirm if your ssh-key is ready.

Now, git clone OSI environment:

Open a terminal

Clone your OSI environment repo:

 $ cd ~
 $ git clone git@github.com:NEU-CS6640-labs/osi_env.git osi

Check the repo:

 $ cd ~/osi/
 $ ls
   <You should see:>
   env.sh

Section 3: Preparing build environment

To run the labs, you will need (1) a cross compiler to compile C code to RISC-V machine code, and (2) a qemu that emulates RISC-V CPUs running on your host OS with x86 or ARM CPUs.

Exercise 1 download compiler, qemu, and setup environment

download compiler and compiling tools:
for Ubuntu and WSL, download this.
for MacOS, download this.
if you’re using other OSes, you’re on your own; here is the tooling page for all supported OSes.
download RISC-V qemu:
for Linux and WSL, download this.
for MacOS, download this.
for others, you’re on your own; see here
move the two downloaded files (named xpack-riscv-....tar.gz and xpack-qemu-....tar.gz) to the osi folder (~/osi)
unzip the downloaded files in ~/osi
$ cd ~/osi/
$ tar -zxvf xpack-riscv-....tar.gz
$ tar -zxvf xpack-qemu-....tar.gz
Now, you should see two new folders in ~/osi with the same name of the downloaded files.
update the environment file:
open ~/osi/env.sh with your favorite text editor.
uncomment the two lines that matches your OS (Ubuntu/WSL or MacOS)
double-check whether the two lines matches the newly created folder names
load environment variables
$ source ./env.sh
// OR
$ . ./env.sh
This command loads egos-2k+ variables to your current shell/terminal. Note:
You need to run this when you open a new shell/terminal.
You should run this only under ~/osi/ because the script defines home folder based on current working directory.
confirm the environment variables are ready
$ echo $EGOS
  <you should see the full path of "osi" folder>
$ echo $PATH
  <you should see the two newly created folders:
   "xpack-riscv-..." and "xpack-qemu-..." in the PATH>
confirm gcc and qemu are installed
$ riscv-none-elf-gcc --version
  <you should see version information of gcc>

$ qemu-system-riscv32 --version
  <you should see version information of qemu>
Note: for MacOS users, if you see permission warnings, you need to grant the permission. See HowTo here.

This completes the part 1.

Part 2: Hello world in egos-2k+

Git clone the egos-2k+ repository

Please click the “egos-2k+” link on Canvas homepage to create your own private clone of the egos-2k+ repository; this clone lives on (is hosted by) GitHub. Once that clone exists, you will perform a further clone to get that private repository onto your working environment (your machine). You’ll do your work on your machine, and then push your work to the GitHub-hosted private repository for us to grade.

Here’s how it should work.

click the “egos-2k+” link on canvas homepage to create your clone on github.
log in to your github.
provide a name.
the link should automatically clone the repository, for instance, if your username name was foobar, you should now have a repository on your github called NEU-CS6640-labs/egos-2k-foobar.

creating a local clone

exercise 2 clone and build egos-2k+

open a terminal
clone your egos-2k+ to osi folder:
```
 $ cd ~/osi
 $ git clone git@github.com:NEU-CS6640-labs/egos-<Your-GitHub-Username>.git egos
```
Note that the git@github.com:... can be obtained on GitHub by clicking the “Clone or download” button. You want to clone using SSH, not HTTPS, so you might need to click “Use SSH”.

Check the repo:

 $ cd ~/osi/egos/
 $ ls

   <You should see:>
   apps            earth           grass           library
   LICENSE         Makefile        slack           tools

Setup stream remote repository
```
 $ git remote add upstream git@github.com:NEU-CS6640-labs/egos-upstream.git
```
Note:
- you will have two remote repos—(1) origin points to your private GitHub repo where you should submit your work (this is also where we pull and grade your labs); (2) upstream points to a repo where we publish labs (this is where you should fetch the code in the beginning of each lab, starting from lab2). You can confirm this by $ git remote -v.

Build egos-2k+

 <in ~/osi/egos/ and have executed "source ./env.sh">

 $ make
 Compile app cat => build/release/user/cat.elf
 ...
 -------- Compile EGOS --------
 ...

Run egos-2k+

 $ make qemu
 -------- Simulate on QEMU-RISCV --------
 ...
 [CRITICAL] Welcome to the egos-2k+ shell!
 ➜ /home/cs6640 %

 <you can try "echo README" and "ls" in egos>

Note: to quit, press Ctrl-a x; that is, press and hold “Ctrl” and press “a”, release both, and press ‘x’.

Now, you can run egos-2k+ on qemu (which simulates a RISC-V CPU). Next, let’s write your first line of code.

Exercise 3 Hello world in egos-2k+

open apps/user/helloworld.c
in main(), write printf("Hello world!\n");

recompile, run qemu, and run helloworld:

 <in ~/osi/egos/>

 $ make qemu
 ...
 ...
 [CRITICAL] Welcome to the egos-2k+ shell!

 <type in "helloworld" in egos-2k+ shell>
 ➜ /home/cs6640 % helloworld

 <you should see:>
 Hello world!

Note: you need to rebuild the OS (namely, make or make qemu) for any of your modification to take effect.

Understanding memory, C pointers, and CPU instructions

Below are three exercises that help you better understand basics of OSes:

Exercise 4: we’re using qemu to emulate a RISC-V CPU. In this exercise, you will use a simple tool to explore this CPU’s valid memory.
Exercise 5: “C pointers are memory addresses”. Is this true? You will confirm this claim in the exercise.
Exercise 6: you will see what “instructions” are—they are 0s and 1s.

Exercise 4 Explore valid memory

read apps/user/memloop.c and understand its behavior
run memloop with 0x02000000 in egos-2k+:
```
 ➜ /home/cs6640 % memloop 0x02000000

 <you should see:>
 input addr[0x02000000] => 2000000
 touch memory start: 2000000
 - touched page(1) [2000000, 2001000)
 ...

 <then, some exception happens and the kernel stops working>
```
Note: kernel will panic and hang in the end because memloop touches a non-existent memory. You need to quit qemu by Ctrl-a x (holding “Ctrl” and pressing “a”, releasing both, then pressing ‘x’).
use memloop to explore the following addresses, and answer questions in slack/lab1.txt:
- 0x10013000
- 0x20000000
- 0x80000000

Exercise 5 C pointers are memory addresses

read ptr_is_address in apps/user/helloworld.c
use the valid memory address you found in Exercise 4 to convince yourself that pointers are addresses.
Remember to call ptr_is_address in main and rebuild (make).
You may see warning messages, weird behaviors, or instant crashing of the kernel when you randomly read/write memory. This is expected, as you may sabotage memory that is important.
find one address that contains 0.
(We expect to see that ptr_is_address prints:
ptr:<picked-address> before:0 after:1)

Exercise 6 instructions are 0s and 1s

read ins_is_zero_one in apps/user/helloworld.c
call ins_is_zero_one in main, and run helloworld
```
 ➜ /home/cs6640 % helloworld

 <you should see:>
 Enter ins_is_zero_one()
```
Now, you’re stuck here because there is an infinite loop in ins_is_zero_one. Use Ctrl-c (press-and-hold “Ctrl” and press “c”) to kill the process.
Next, you will need to replace the instruction at memory 0x8020000C—the instruction that runs when pressing Ctrl-c—so that, instead of exit, you will re-enter main function.
Most of the work has been done by skeleton code. You need to do one thing: assign call_main with the binary (it will be 4B) of call main (this is an RISC-V assembly instruction).
How to find the binary form of this instruction? Search call main in build/debug/helloworld.lst.
If succeeded, you should see:
```
 ➜ /home/cs6640 % helloworld
 Enter ins_is_zero_one()
 <press Ctrl-c>
 Enter ins_is_zero_one()
 <press Ctrl-c>
```
Pressing “Ctrl-c” will not kill the process because the code will jump to call main again. (Now, use Ctrl-a x to quit qemu.)

Finish implementing `printf`

Next, you will complete the implementation of printf, which will be used extensively in future labs.

Exercise 7 implementing printf()

read format_to_str function in library/libc/print.c and understand how printf processes formatted inputs.
add support for the %c, %u, %p, %lld, %llu, and %llx format specifiers. Google their definitions if needed.
read man va_arg to understand how variadic arguments are retrieved.
For numeric conversions, implement helper functions similar to itoa and use existing standard C library facilities if needed.
Comment out function calls from prior exercises in main.
Test your implementation by uncomment and run impl_printf() in apps/user/helloworld.c Use a calculator to verify that the outputs are numerically correct.

This completes the part 2.

Finally, submit your work

Submitting consists of three steps:

Executing this checklist:
- Fill in ~/osi/egos/slack/lab1.txt with (1) your name, (2) your NUID, (3) GitHub id, and (4) slack hours you used.
- Make sure that your code build with no warnings.

Push your code to GitHub:

 $ cd ~/osi/egos/
 $ git commit -am 'submit lab1'
 $ git push origin

 Counting objects: ...
  ....
  To ssh://github.com/NEU-CS6640-labs/egos-2k-<username>.git
   7337116..ceed758  main -> main

Actually commit your lab (with timestamp and git commit id):
1. Get the git commit id of your work. A commit id is a 40-character hexadecimal string. You can obtain the commit id for the last commit by running the command git log -1 --format=oneline.
2. Submit a file named git.txt to Canvas. (there will be an assignment for this lab on Canvas.) The file git.txt contains two lines: the first line is your github repo url; the second line is the git commit id that you want us to grade. Here is an example:
```
 git@github.com:NEU-CS6640-labs/egos-2k-<username>.git
 29dfdadeadbeefe33421f242b5dd8312732fd3c9
```
  Notice: the repo address must start with git@github.com:... (not https://...). You can get your repo address on GitHub repo page by clicking the green “Code” button, then choose “SSH”.
3. Note: You can submit as many times as you want; we will grade the last commit id submitted to Canvas. Also, you can submit any commit id in your pushed git history; again, we will grade the commit id submitted to Canvas.
  Notice: if you submit multiple times, the file name (git.txt) changes to git-N.txt where N is an integer and represents how many times you’ve submitted. We will grade the file with the largest N.

NOTE: Ground truth is what and when you submitted to Canvas.

A non-existent repo address or a non-existent commit id in Canvas means that you have not submitted the lab, regardless of what you have pushed to GitHub—we will not grade it. So, please double check your submitted repo and commit id!
The time of your submission for the purposes of tracking lateness is the timestamp on Canvas, not the timestamp on GitHub.

This completes the lab.

Acknowledgments

Some instructions in Part1 are copied from CS5600, which are further borrowed from from Mike Walfish’s CS202, with materials from Harvard’s CS61, Jinyang Li’s CS201, and Aurojit Panda’s 3033. Some shell commands are borrowed from Yunhao Zhang’s egos-2000 USAGE page.

Appendix

What is git?

Git was developed by Linus Torvalds for development of the Linux kernel. It’s is a distributed version control system, which means it supports many local repositories which each track changes and can synchronize with each other in a peer-to-peer fashion. It’s the best widely-available version control system, and certainly the most widely used. For information on how to use git, see:

For the workflow in GitHub:

GitHub Guides: Hello World

Teaching GitHub about your identity

The easiest way to access GitHub repositories is using an SSH key, a secret key stored on your machine that defines your identity. Follow the steps below to create a key for your machine.

Open Terminal.
Run $ ssh-keygen -t rsa -b 2048 (don’t copy the $ sign) and follow the instructions.
- Press enter to use the default file path and key name (should be ~/.ssh/id_rsa).
- Choose a password or leave it empty.
This creates your ssh keys, which live in the directory ~/.ssh. Your public key is in the file ~/.ssh/id_rsa.pub.
Run cat ~/.ssh/id_rsa.pub to display your public key.
Copy your public key (that is, select the text on the screen, and copy it to the clipboard).
In GitHub, go to your profile settings page (accessible via the upper-rightmost link–this looks like a bunch of pixels for new accounts). Select “SSH and GPG keys” and hit the “New SSH key” button. Then copy and paste the contents of your ~/.ssh/id_rsa.pub (from your machine) into the “Key” section. Give the key a sensible title, hit the “Add SSH key” button, and you’re good to go.

(if you haven’t) configure your git “identity”:

 $ git config --global user.name "FIRST_NAME LAST_NAME"
 $ git config --global user.email "YOUR_@COLLEGE_EMAIL"