Welcome to the Devilry user documentation¶
User documentation for Devilry.
Note
You can contribute to these docs. Use the Show on GitHub or Edit on GitHub (if logged into github) links in the sidebar.
You can also submit issues with these docs to the devilry-userdoc issue tracker.
Getting started¶
We recommend that you start with a quick look at Common concepts. This avoids confusion with a few special terms that Devilry uses to allow it to be used in many different settings. When you have skimmed over the concepts, you can continue with the other Topic guides below.
More help¶
If this documentation is lacking, or if you have problems, detect bugs, etc...
Forum, issue-tracker and contact information¶
Note
Devilry is truly open, not just Open Source, but we also try to keep all issues, suggestions and plans in the open. This means that your suggestions, bugs, problems, etc. is handled in the open, and readable by anyone.
Warning
DO NOT post sensitive information, like names of students, passwords, etc. via any of the contact channels listed below (see the note above for why).
Issue tracker¶
Visit The Devilry issue tracker.
Anyone can add issues to our issue tracker at our GitHub project page. We use the issue tracker for bugs, problems, suggested improvements, suggested new features, etc.
You need to create a GitHub user to add an issue. You just have to write an understandable title and description. We will then tag your issue, and respond to your via comments on your issue. You should be notified for each new comments on your issues by email unless you disable email notifications on GitHub.
Question and Answers forum¶
Visit: The Devilry Help Questions and Answers forum.
We have a Question and Answer forum on Google Groups named Devilry Help. You can post anything on this forum, including:
- help understanding Devilry — no problem is too small for this forum
- suggest improvements — even minor improvements
- report problems
- report bugs
- suggest new features
Note
We recommend that you use the issue tracker instead of this forum if you have a well defined problem or suggestion. Even very small improvements or issues belong in the issue tracker, and they end up there even if you post them in the Q&A forum. The only difference is that someone else have to put them in the issue tracker, which may delay fixing the issue.
Your local Devilry support¶
The local Devilry support typically adds a link to a page with their contact information in the Devilry header. Click your name in the upper right corner when logged in to Devilry, and see the links in the Help-section.
Contact email — only for special cases¶
Warning
This is only for contact requests that does not belong in the open contact channels, like the issue tracker, or in the Q&A forums. This mailinglist is typically for those that need private and direct contact with the Developers, and requests belonging in the other contact channels is ignored.
The warning above about sensitive information is also for this list.
Common for all roles¶
Common concepts¶
Localization¶
Devilry can be translated to multiple languages and dialects. Each dialect makes Devilry feel more at home in its environment. A dialect translates special terms (see Special terms and concepts). This is important to understand when reading this documentation, because we use the special terms throughout the documentation, not the dialects.
Special terms and concepts¶
Devilry has some special terms and concepts. The most important (that cause most confusion) is:
- Period: A range of time. Typically a semester or a year.
- Examiner: Someone that provides feedback.
- Group: Students are always in a (project) group even when they work alone on an assignment.
More details about these and more terms and concepts follows below.
Node¶
A Node is a place to organise top-level administrators (I.E.: administrators responsible for more than one Subject (course)). Nodes are organised in a tree. This is very flexible, and can be used to emulate most administrative hierarchies. A node is often a department, or some other organizational unit, but the exact use in your local Devilry instance depends on how you choose to organize administrators in Devilry.
Subject (course)¶
A subject is, as far as Devilry is concerned, a container of Periods. In a typical Devilry setup, a Subject is the same as a Course, and each Period within the Subject is a semester or year.
Period (semester, year, ...)¶
A period is a limited span of time (I.E: january to july 2011) that you give a name (I.E.: Spring 2011). You register assignments on a period, and register students and examiners on each assignment.
Group, Candidate and Student¶
Students are not registered directly on an assignment. Instead a group is created, and one or more students is added as Candidates on that group. This means that project assignments, where students cooperate, is organized exactly like any other assignment. The only difference is the number of Candidates in each group.
A Candidate can also have a candidate ID, which is used to identify the student on anonymous assignments like exams.
See also
Deadline¶
Deadlines are individual for each group. They are organized below a Group in the Devilry hierarchy. In other words: Each Group has one or more deadlines.
Examiner¶
Examiner is someone that writes feedback. Examiners are often one of these:
- A teacher that corrects their own students. They are usually Period or Subject administrator in addition to Examiner.
- A teaching assistant.
- Someone giving anonymous feedback on an exam.
A user becomes examiner when they are assigned as examiner for a group (See Group, Candidate and Student) by an administrator.
See also
Special terms in context — a typical Devilry hierarchy¶
The tree below is an example of a typical Devilry hierarchy for a university named Duckburgh University with the special terms in brackets.
- Duckburgh University [Node]
- Department of Physics [Node]
- PHYS 101 — Introduction to physics [Subject (course)]
- Spring 2011 [Period (semester, year, ...)]
- Assignment one
- Peter Pan and Wendy [Group, Candidate and Student]
- Deadline feb. 27 2012 19:30 [Deadline]
- Delivery 1
- Captain Hook [Group, Candidate and Student]
- John Doe [Group, Candidate and Student]
- Deadline feb. 25 2012 23:35 [Deadline]
- Delivery 1
Spring 2012 [Period (semester, year, ...)]
Spring 2013 [Period (semester, year, ...)]
PHYS 302 — Advanced physics [Subject (course)]
...
- Department of Informatics [Node]
- INF 101 — Introduction to programming [Subject (course)]
- INF 102 — Objectoriented programming [Subject (course)]
- ...
...
Email sending in Devilry¶
Email sending is highly configurable in Devilry. This guide deals with the default configuration.
When does Devilry send email?¶
Whenever a student makes a delivery, and whenever they get a new feedback.
How to test email sending (for system admins)¶
Go to the frontpage, select the Superuser role, select Users, select a user and click Send a test-email to USERNAME in the upper right corner.
Student¶
The Student role¶
The student interface should be intuitive to use. Please post an issue with our Issue tracker if that is not the case.
How to create project groups (collaborate on an assignment)¶
How it works¶
Project groups are created on a per assignment basis. Features:
- Any member of the group can make deliveries on behalf of the group.
- Any feedback given to the project group is for all group members. If you are supposed to get individual feedback, you should not be in a project group.
- When you join a group, you get any deliveries and feedback made by the group before joining, and they get any deliveries and feedback you may have had when joining the group.
- When you leave a group, you do not loose any deliveries and feedback. You even keep deliveries and feedback made before you joined the group.
Invite other students to join your group¶
If your course administrator have enabled collaboration, you can invite other students to join your group as follows:
- Log in to Devilry.
- Select the Student role on the frontpage.
- Select Active semester in the Navigate section at the bottom of the page.
- Select the assignment.
- You should have a Group members section in the top of the left hand column with the current group members, and an Invite students to the group link. Click the link.
- Use the wizard to send an invite. Other students will get their invite via email, and they can accept or decline the invite. You can delete an unanswered invite.
Warning
All students you invite to your group will given all current deliveries and feedback, even deliveries and feedbacks made before they joined the group. If they leave the group, they will keep all deliveries and feedback you have received on the assignment, even feedback and deliveries made before they joined the group.
Note
Refer your course administrator to subjectadmin_groups if you think they should enable collaboration.
Leave a group / kick a member¶
You need to ask a course administrator if you want to leave a group or kick a group member. Leaving a group or kicking a member is perfectly safe. Any member leaving a group is simply moved into a complete copy of the group including all deliveries and feedback. The only difference is that the original group gets one less member, and the new group will only have one member.
Examiner/Corrector¶
The Examiner role¶
Note
Examiner is someone that writes feedback. Examiners are often one of these:
- Someone responsible for correcting some or all students in a subject/course.
- A teacher that corrects their own students. They are usually Period or Subject administrator in addition to Examiner.
- Someone giving anonymous feedback on an exam.
Getting started¶
Getting started guide for examiners¶
Note
To avoid confusion when reading this guide, please read Common concepts, at least the Group, Candidate and Student section.
Note
This guide is under construction. Please contact devilry-support@ifi.uio.no if you have questions of any kind related to Devilry.
Choose the examiner role¶
After successful login you need to choose Examiner from the list of available roles.
Select an assignment¶
On the examiner dashboard, assignments where you are examiners are listed. To start correcting, you must choose the spesific assignment in which you want to work with. The list are ordered by the time the assignments where published in descending order. This ensure that your latest assignment are listed on top.
Select the group to correct¶
After the assignment is choosen you are redirected to a view that list all groups on the assignment.
The list may be filtered to show a more fine grained selection:
- Waiting for feedback
- Shows the assignment groups that have at least one valid delivery and are waiting for feedback to be provided by the examiner.
- Waiting for deliveries
- Shows the assignment groups that have not provided a valid delivery yet.
- Corrected
- Shows the assignment groups that are corrected. Be aware that Corrected will list both failed and passed groups as long as they are corrected.
- All
- Shows every assignment group on the assignment where yourself are the assigned examiner. This is the default filter upon entering the view.
To start correcting you would normally filter with the Waiting for feedback option. The Write feedback button on the group item in the list will take you directly to the latest delivery provided by the group. see Add feedback to the delivery for further details.
If you need access to older deliveries provided by the group you must click on the group name header in the list item which redirect you a detailed view for that assignment. More information on this view in Correct an earlier delivery
Add feedback to the delivery¶
Upon entering the delivery you will need to click the Provide feedback button to start giving feedback.
This view may vary depending on the grading system configured for the assignment. The picture show a simple points system with a form to provide the number points achieved.
Every grading system features the ability to provide a feedback text in addition to the grade information. Click inside the textbox and a WYSIWYG Markdown editor will be shown. Just type in the feedback and push the Publish button to publish the feedback without examine a preview. If you lack experience with Markdown you would probably want to click the Preview button to be able to secure that the feedback appear as supposed.
You may save you work for later by clicking the Save Draft button
Correct the next delivery¶
When there is multiple groups Waiting for feedback you can loop through each group by clicking the arrow button in the upper right corner of the view.
Correct an earlier delivery¶
When choosing the spesific group every delivery attempts will be listed categorised by the their respective deadlines. This makes it possible to correct a delivery that is not the latest one.
Other examiner guides¶
How to correct several groups at once.¶
Note
To avoid confusion when reading this guide, please read Common concepts, at least the Group, Candidate and Student-section.
Introduction¶
This guide show you how to edit feedback for a selection of groups. To get to the view described here you must first choose the examiner role in the Devilry frontpage and then choose an assignment.
Editing of multiple groups is easy and straightforward. Activate the checkboxes next to the group of interest to add it to your selection. If you want to choose all groups within the active filter you may click Select All button to speed up the process. When you have created your selection click the Write feedback button to start providing feedback.
Provide feedback¶
The groups that you selected are listed after the Edit feedback for: text, please make sure that it is complete and correct before you take further actions. Provide your feedback. Click Preview button to get a glimpse on how your feedback will look when completely rendered. Be aware that Preview will save your feedback draft on each group. You will be able to come back and edit further in the next step.
If you want to publish your work right away just click Publish.
Manage deadlines — for Examiners¶
Add a deadline¶
When an assignment is created, an initial deadline is set. You can not change this deadline, however you can extend the deadline by creating a new deadline. You usually create deadlines for the following reasons:
A student gets a failing grade¶
If you want to give the student a change to get a new attempt Devilry will ask you (when you publish a failing feedback) if you wish to give the group a new deadline.
Move a deadline¶
Currently, examiners can not move deadlines. This is coming along with an update of the examiner UI. For now, you have to ask an administrator if you need to move a deadline.
How to add feedback to a student without deliveries¶
Note
This guide is a supplement to Getting started guide for examiners.
Choose the examiner role¶
It should be one of the available roles on your frontpage.
Select an assignment¶
On the examiner dashboard, assignments where you are examiners are listed ordered by publishing time in descending order. Choose an assignment from this list to get start giving feedback on that assignment.
Select a group without any deliveries¶
Select a group that waiting for feedback, but does not have any deliveries, as illustrated in the figure below:
Click the “add non-electronic delivery” button¶
If the group: - Has no deliveries. - Is waiting for feedback (their last deadline has expired).
You will see a orange warning-box like the one in this illustration:
Click the add non-electronic delivery button, and follow the instructions on the next page.
Devilry flavoured Markdown¶
When you write feedback to your students, you use the Markdown text formatting language.
With Markdown, you to write using an easy-to-read, easy-to-write plain text format, and let someone else (Devilry) worry about how the results will look. This makes it possible to write feedback text that Devilry can optimize for anything from smartphones to large desktop displays.
Basics¶
Paragraphs are just one or more lines of consecutive text followed by one or more blank lines:
Maecenas faucibus mollis interdum. Vestibulum id ligula porta felis euismod
semper. Vestibulum id ligula porta felis euismod semper. Aenean lacinia
bibendum nulla sed consectetur.
Donec id elit non mi porta gravida at eget metus. Vestibulum id ligula
porta felis euismod semper. Praesent commodo cursus magna, vel scelerisque
nisl consectetur et.
# Largest heading
## Second largest heading
...
##### Very small heading
*Italic text*
**Bold text**
Check out [http://devilry.org](The devilry website).
Unordered lists (bullet lists):
* This
* is
* a
* test
Ordered lists (numbered lists):
1. Item one
2. Item two
3. Item three
As stated on the first page of the 101 guide:
> You have to learn to walk before you can learn how to run
Advanced¶
If you want to use a special Markdown character in your document (such as displaying literal asterisks), you can escape the character with a backslash. Markdown will ignore the character directly after a backslash. Example:
This is how the \_ (underscore) and \* asterisks characters look.
Devilry Markdown supports LaTeX math through the MathJAX library/renderer. Examples:
A simple example:
$mathblock$
^3/_7
$/mathblock$
The Lorenz Equations:
$mathblock$
\begin{aligned}
\dot{x} & = \sigma(y-x) \\\\
\dot{y} & = \rho x - y - xz \\\\
\dot{z} & = -\beta z + xy
\end{aligned}
$/mathblock$
You have to escape special Markdown characters such as \, which is why we have \\\\ at the end of our lines in the example above instead of just \\.
You can easily show syntax highlighted code blocks:
Java code:
``` java
class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello world");
}
}
```
Python code:
``` python
if __name__ == "__main__":
print "Hello world"
```
C code:
``` c
#include<stdio.h>
int main() {
printf("Hello World");
return 0;
}
```
C++ code:
``` c++
#include <iostream>
int main() {
std::cout << "Hello World!";
return 0;
}
```
HTML example:
``` html
<html>
<body>
<h1>Hello world</h1>
</body>
</html>
```
CSS example:
``` css
body {
background-color: pink;
color: green;
font-size: 80px;
}
```
Any code:
```
for x in 1 through 3
show x
```
Devilry supports all languages supported by Pygments.
Frequently asked questions¶
Examiner UI FAQ¶
- How can I set a new deadline?
- When the deadline is up you will be asked if you want to provide a new deadline for students to deliver.
What can an examiner do?¶
Examiners can basically do anything non-destructive with groups (see Group, Candidate and Student) that they have been assigned to:
- View all feedback for the groups.
- Add new deadlines to their groups.
- Give feedback.
- Change feedback. Each change to the published feedback is logged, and the students can view all published feedback.
Examiners can not:
- Delete groups.
- Add new groups.
- Add or remove students to groups.
- Move deadlines. This is a missing feature, which will be implemented in a future release. It will be safe becuase students will remain protected because of logging of all changes. Moving deadlines is already implemented for administrators.
- Remove deadlines from groups. We will probably allow examiners to remove deadlines without deliveries when we start allowing them to move deadlines.
Subject/Course administrator¶
Note
For users managing one or more subjects/courses.
Introduction to the Subjectadmin role¶
Note
Please read, or at least take a quick look at, the Common concepts before reading this guide.
Who this guide is for¶
A subject is, as explained in the common concepts guide, typically a course. This means that this guide is for administrators managing a course or a period (semester) within a course. If you have yellow background color in your header, you are using the user interface for the Subjectadmin role.
The responibilities of a Subjectadmin¶
A Subjectadmin manage one or more Subject (course), and/or Period (semester, year, ...). They set up assignments, organize students into Groups, and assignment Examiners to give feedback to students.
Commmon tasks¶
Finding the Subjectadmin frontpage¶
The Subjectadmin frontpage is the page that you navigate to when you select the Subject administrator role (may be something like Course administrator in your local dialect), from the Devilry frontpage.
Create an assignment¶
We provide an interractive guide to help you create assignments. Simply go to the Subjectadmin frontpage (see Finding the Subjectadmin frontpage), and select the guide on the right hand side.
Get an overview over all your students¶
Select an active Period (semester, year, ...) from the Subjectadmin frontpage (see Finding the Subjectadmin frontpage), or an old/expired Period (semester, year, ...) via the link further down on the frontpage.
Under the Edit and view related information heading, you will find links to your Students, and a link to an Overview of all results. You can export (download) the overview as MS Excel, CSV, and a couple of other formats. The export-links is in the toolbar right below the page heading.
Manage deadlines — for Subjectadmins¶
Subject administrators can create, view, move and edit deadlines using the Deadlines link on the main page for an assignment.
Overview of the deadline view¶
The image below is a typical deadline overview. Each of the red-marked areas is explained below the image.
Add deadline (hilighted with a red circle)¶
Click the Add deadline button to a add deadline to one or more groups. You can select individual groups, and Devilry has shortcuts for a couple of common choices.
Expand/collapse deadline (hilighted by a red arrow)¶
Deadlines start out collapsed when you enter a view. Click the deadline to expand or collapse it.
Move/edit deadline (hilighted by a red rectangle)¶
To move or edit (the about-text for) a deadline, expand the deadline, and select the edit-button.
You can change the deadline for only some of the groups within the deadline, effectively splitting the deadline in two. This is explained on the right-hand side of the list of groups that appear when you choose that option.
Delete deadline (hilighted by a red rectangle)¶
You can delete a deadline, but only superusers can delete deadlines where groups have made deliveries.
Hard VS soft deadlines¶
Soft or hard deadlines is configured on the left hand side in the assignment overview under the Deadline handling heading. Use the More info button to get detailed information about soft and hard deadlines.
What about examiners?¶
Examiners can add deadlines. It is a natural part of their workflow. Whenever they fail a student, they are asked to do one of the following:
- Leave the student with a failing grade.
- Give them another chance — create a new deadline.
How to administer project groups (students that collaborate)¶
Concepts and features¶
Devilry is designed with cooperative deliveries in mind.
- Students are always in a group even when they work alone.
- Course admins can safely create and split up project groups at any time.
- You can enable students to create project groups on their own.
- Groups are created and managed per assignment. This means that changing a group on one assignment does not affect that group on another assignment. You can copy groups from another assignment when you create a new assignment.
- Students can be organized in groups even after they have made deliveries and been given feedback. They their respective groups are simply merged into a single group with all deliveries and feedback. The last feedback (if any) is made the active feedback.
How to enable students to create project groups on their own¶
If you want to allow students to form project groups on their own, you have to enable this option on the assignment:
- Go to the overview page for an assignment where you have administrator rights.
- Click any of the edit buttons in the sidebar to your left, except the edit button for Grading system.
- Edit the options in the Allow students to form project groups section.
Note
To see how students form project groups on their own, see How to create project groups (collaborate on an assignment). You should refer your students to that guide when you ask them to form their own groups. Devilry does not notify students when you enable this feature.
Note
This is not as intuitive as it should be. It will be made more intuitive in the future.
How to manually create a project group¶
- Open the students overview on the relevant assignment.
- Select two or more groups/students.
- Select Create project group.
Exactly what this means is explained when you click Create project group, and you have to confirm before the group is created. In short,
How to remove a student from a group¶
Students can not leave groups on their own (yet). So an admin has to manage that:
- Open the students overview on the relevant assignment.
- Select the group.
- Click the red minus button on the right hand side of the student you wish to remove from the group.
This will do the following: - Create a copy of the group with all deliveries and feedback, even
deliveries made by other students before the student you are removing joined the group.
- Add the student you are removing to the copy of the original group.
Examiners — How to set examiners, and how to create feedback¶
How to give feedback to students when you are Subjectadmin¶
Starting with Devilry 1.2.1 it is no longer possible to give feedback using the Subjectadmin role. You have to make yourself examiner.
How to make yourself examiner¶
The easiest way of making yourself examiner is to make yourself examiner when creating a new assignment. If you are one of many examiners, you will have to make sure you are tagged appropriately. The Create new assignment wizard helps you with this when you get to that step.
The other way of making yourself examiner is to do it manually after the assignment has been created. Select an assignment, and select the Students-link. The see the help-column for more help.
How to make others examiner¶
This is basically the same as the previous section. Just choose other users than yourself.
What do examiners have permission to do¶
Note
The reason why it is no longer possible to provide feedback as Subjectadmin is that we have need to be able to optimize the workflows for examiners and subject admins independently. Mixing the roles leads to confusion in all but the most simple cases, and it increases the development time required for each change to any of the user-interfaces significantly.
Choose students that qualify for final exams¶
Quickstart¶
We have an interractive guide for this on your right-hand side on the Subjectadmin frontpage (see Finding the Subjectadmin frontpage). Select the guide, and follow its instructions.
How the qualified for final exams system works¶
You select the students that qualify for final exams using one of the provided plugins. Nodeadmins, or automatic exporters, read these lists to determine who can participate in the final exams.
Students can see if they are qualified for exams or not, but they can not se why they are qualified (they can not see what plugin was used, and with what settings). You should use the course website, or other approprite channels to inform your students about the requirements for final exams.
Changing or retracting¶
You can retract or change a saved qualified for final exams-status.
Warning
Nodeadmins are not notified when you retract or change a status. We are working on a system that handles updates/retracting, but that did not make it into the first release of the Qualifies for final exams app.
Please notify the people coordinating final exams for your department/organizational unit if you change a status.
To change a status, simply use the button at the bottom of the box at the top of the page showing a status.
No plugin fits my needs!¶
Contact the Devilry developers and we will try to help you.
Node/Department admin¶
Note
For users managing a Node containing multiple subjects/courses.
Introduction to the Nodeadmin role¶
Note
Please read, or at least take a quick look at, the Common concepts before reading this guide.
The responibilities of a Nodeadmin¶
A Nodeadmin manage one or more Node. They typically have responsibility for an entire department or organizational unit, where a node represents a department/organizational unit.
Finding the Nodeadmin frontpage¶
The Nodeadmin frontpage is the page that you navigate to when you select the Node administrator role (may be something like Department administrator in your local dialect), from the Devilry frontpage. The button is orange.
Find students, subjects, etc¶
We recommend that you use search to find items in Devilry a a Nodeadmin. You can find the search-panel in the header on the left hand side of your name in the upper right corner of any page.
View/browse students that qualify for final exams¶
To view/browse students that qualify for final exams, grouped by their subject/course, you need to:
- Finding the Nodeadmin frontpage.
- Navigate to a Node containing subjects/courses.
- Select the Qualifed for final exams-link in the Tools section.
Please read¶
The guide for subject/course admins, to know how they interract with the Qualified for final exams system.