Author: Stan Eisenstat
Subject: Comments on the grading of fiend
Date: Tuesday, 29 Sep 2020, 21:52:31
The final test script for fiend is Hwk1/final.fiend and I have posted
gradesheets to your submit directories. Below are
* Comments on grades in general
* Statistics for Homework #1 fiend
* Using see.grades and gross.grades to see your grades
* Comments on fiend, including how to retrieve your gradesheet
* How the grading script works
--Stan-
Comments on Grades
~~~~~~~~~~~~~~~~~~
From a purely mathematical perspective the grades on the ideal homework
assignment or exam should range from a low of 0% to a high of 100% with a
median of 50%. This gives the most information about how well each student
performed. It is also unacceptable psychologically!
Thus my goal is a more modest 60% to 100% range with a median of 80%, which
seems acceptable on both counts. As you will see below, I almost succeeded
with the final script for fiend.
But I also want to give final grades that are more or less consistent with Yale
norms. Thus I have the following rules for letter grades:
1) To give at least 45% A's and A-'s, the lower limit on the A/A- range will be
low enough that at least 45% of the class lies above it. There are similar
rules for the other grade ranges.
2) To honor the expectation that a student whose total grade is at least 90%
should receive an A or A-. Thus the lower limit on the A/A- range will be
at most 90% (even if this means giving more than 45% A's and A-'s). There
are similar rules for the other grade ranges.
3) The PASS/FAIL line is around 55%, after all lateness penalties are ignored.
Finally, fiend took the median student about the same amount of time as I was
expecting based on past versions of this assignment. History should take
significantly less time.
Statistics for Homework #1 fiend
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#Tests Passed
~~~~~~~~~~~~~
min = 32.00, max = 60.00, median = 50.00, avg = 49.12
58 < X <= 60: ( 2) X X
56 < X <= 58: ( 6) X X X X X X
54 < X <= 56: ( 7) X X X X X X X
52 < X <= 54: ( 8) X X X X X X X X
50 < X <= 52: ( 8) X X X X X X X X
48 < X <= 50: ( 7) X X X X X X X
46 < X <= 48: (10) X X X X X X X X X X
44 < X <= 46: ( 5) X X X X X
42 < X <= 44: ( 3) X X X
40 < X <= 42: ( 2) X X
38 < X <= 40: ( 4) X X X X
36 < X <= 38: ( 2) X X
X <= 36: ( 3) X X X
Estimated Time
~~~~~~~~~~~~~~
10.00 10.00 10.00 15.00 15.00 15.00 15.00 15.00 15.00 16.00
16.00 16.00 18.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00
20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00
20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00 20.00
20.00 23.00 24.00 24.00 25.00 25.00 25.00 25.00 25.00 25.00
25.00 25.00 25.00 25.00 25.00 25.00 30.00 30.00 30.00 30.00
30.00 30.00 30.00 39.00 40.00 40.00 40.00 40.00
Median = 20.00
Total Time
~~~~~~~~~~
11.25 12.17 13.00 14.08 14.50 15.00 15.00 15.75 16.00 17.00
17.83 18.00 18.25 18.50 19.00 19.00 19.50 20.00 20.50 20.67
20.67 21.00 21.33 21.50 21.50 22.00 22.67 22.67 22.75 23.00
23.42 23.75 23.83 24.00 25.00 25.00 25.00 26.00 26.00 26.50
27.50 27.50 27.50 28.00 28.50 28.67 29.00 30.00 30.50 31.00
31.33 31.83 32.00 32.00 33.00 33.00 33.50 34.50 36.08 38.00
39.50 43.50 48.00 48.00 49.75 55.50 68.00 76.00
Median = 24.50
see.grades and gross.grades
~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are two shell scripts in /c/cs323/bin:
1. see.grades prints out a list of your grades on each of the assignments (as
they appear on the master list) and tells you the number of students with
strictly lower and strictly higher grades on each assignment.
2. gross.grades does the same thing except that all lateness penalties are
ignored.
Please let me know if you have any problems with see.grades and gross.grades or
if any of the entries seem incorrect.
WARNING: The number of lower grades includes students who are still on the
official grade list but have yet to submit any working code.
Comments on fiend
~~~~~~~~~~~~~~~~~
1. See below for a complete description of how the grading script works. You
can retrieve your grade sheet using the command
% /c/cs323/bin/retrieve 1 GradeSheet
All deductions from the number of tests passed are noted at the end of that
file. Lateness penalties are assessed separately.
2. If your source file(s) did not contain your name and netID, I deducted 0
points if both were missing and 0 points if only the name or netID was
missing, when I entered the score in the grade file. In the future I will
deduct 1 and 0.5 points, respectively.
3. If your log file did not contain a statement of major difficulties, I
deducted up to 1 point (depending on the extent to which your session log
contained such information) when I entered the score in the grade file.
4. Your statement of major difficulties should have been 100 words or longer
(i.e., at least eight 80-character lines, excluding the listing of those
with whom you discussed the solution) and should include assignment-specific
details. In the future I will deduct 1 point for statements that are
significantly shorter or non-specific.
5. If your code did not make, I created a Makefile.CORRECTED and deducted 1
point when I entered the score in the grade file.
6. You can save time by stopping and seeking help when you run into problems
that you cannot resolve in 30 minutes or so. Starting early (hint!) means
that you are less likely to be near the due date when this happens.
If you spent 49 hours or more on the assignment (the median was 24.5 hours),
you may want to make an appointment to see me for ways to reduce the time.
7. If any part of a specification is unclear or confusing, please let me know
so that I can correct it.
8. When debugging, it helps to have small examples where code fails, since it
is easier to trace the flow of execution using a debugger, print statements,
or with a listing.
9. Valgrind is your friend. If you do not understand what it is telling you,
see http://zoo.cs.yale.edu/classes/cs323/doc/Valgrind for useful options and
for common errors and possible explanations of their origins.
A. Please use the unsubmit command to remove any files that are not part of
your ultimate solution.
B. The class web page has links to scatter plots of #Passed vs #Lines, #Passed
vs Time, and Time vs #Lines. Please let me know if you see any trends in
this data.
C. So few students passed Tests #31 that I made it non-credit and replaced it
with Test #61.
How the Grading Script Works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Before the grading script ran the final test script it ran "make fiend"
using the -std=c99, -pedantic, and -Wall options (whether your makefile
specified them or not). If gcc issued any warnings, it printed the message
checkwarn: warning messages from gcc -c -std=c99 -pedantic -Wall
under the header
***** Checking for warning messages *****
near the start of the grade sheet, and I deducted 1 point from the score
that appears on the grade sheet when entering that score in the grade file.
2. After running the tests, the grading script checked for files submitted
after the due date, printing the number of late days and a listing of all
files and their submission dates if any file was late. If you submitted a
LATENESS file, it adjusted the number of late days accordingly. The
lateness penalty (5% per day or part thereof, excluding the week when the
clock was stopped) was applied when I entered the score in the grade file.
3. Next the grading script looked for a makefile. If it did not find one, it
printed a warning message
checkmake: no makefile
under the header
***** Checking makefile *****
near the end of its output, and I deducted 1 point from the score that
appears on the grade sheet when I entered that score in the grade file.
4. Next the grading script printed the submission logs for your file(s)
(excluding the log file and the makefile), showing the revision numbers,
the times of submission, and how many lines were added and deleted. If
the ratio of the number of hours reported to the number of submissions was
sufficiently large, or if the number of lines in an initial submission or a
lower bound on the number of new (not just changed) lines in some update was
sufficiently large, it printed a warning message:
checkrcs: too few submissions: NN in HH.HH hours
or
checkrcs: too few resubmissions: large submission(s)
under the header
***** Checking RCS files *****
near the end of its output. In extreme cases I deducted up to 1 point from
the score that appears on the grade sheet when entering that score in the
grade file. In the future I will both raise the 1-point cap and deduct
points in less extreme cases.
5. Next the grading script searched your log file for the one line that
contains the string "ESTIMATE" and your estimate of the time required made
prior to writing any code. If it did not find such a line, it printed a
warning message:
checklog: no line with ESTIMATE
under the header
***** Checking log file *****
near the end of its output, and I deducted 0.5 point from the score that
appears on the grade sheet when I entered that score in the grade file.
If it did not find such a line even when case was ignored, it printed a
warning message
checklog: no estimate of time spent
and I deducted 0.5 point from the score.
If it found more than one line containing the string "ESTIMATE", it printed
a warning message:
checklog: multiple estimates of time spent
and I deducted 0.5 point from the score.
6. Next the grading script searched your log file for the one line that
contains the string "TOTAL" and the total time that you actually spent prior
to writing any code. If it did not find such a line, it printed a warning
message:
checklog: no line with TOTAL
under the header
***** Checking log file *****
near the end of its output, and I deducted 0.5 point from the score that
appears on the grade sheet when I entered that score in the grade file.
If it did not find such a line even when case was ignored, it printed a
warning message
checklog: no total of time spent
and I deducted 0.5 point from the score.
If it found more than one line containing the string "TOTAL", it printed a
warning message:
checklog: multiple estimates of time spent
and I deducted 0.5 point from the score.
7. Finally the grading script may have printed a list of certain files,
including:
* A file named Makefile.CORRECTED, which means that your code did not make.
In such cases I deducted 1 point from the score that appears on the grade
sheet when I entered that score in the grade file.
* A file named LOG.CORRECTED, which means that your log file did not have an
estimate and/or a total in the proper format. The deduction(s) were taken
as above, and where possible these values were taken from the corrected
log so that they could be reported.
* A directory named Overwritten, which means that you overwrote some of
the files you submitted with those from a later assignment and I had to
restore them.
--Stan-
PREV
INDEX
NEXT