QuizQuestions

QuizQuestions allows the inclusion of self-grading quiz questions within a Documenter, Weave, quarto, or Pluto HTML page.

A few examples:

Basics

The basic idea is:

load the package:

using QuizQuestions
using LaTeXStrings # helper for using math notation

create a question with Julia code:

choices = ["one", L"2", L"\sqrt{9}"]
question = "Which is largest?"
answer = 3
radioq(choices, answer; label=question, hint="A hint")

Which is largest? 🎁

$\sqrt{9}$

one

$2$

repeat as desired.

The quizzes are written in markdown with the questions in Julia blocks. The above code cells would be enclosed in triple-backtick blocks and would typically have their contents hidden from the user. How this is done varies between Documenter, Weave, quarto, and Pluto. The examples directory shows examples of each.

For each question:

The show method of the object for the text/html mime type inserts the necessary HTML and JavaScript code to show the input widget and grading logic.
the optional hint argument allows a text-only hint available to the user on hover.
The optional label argument is used to flag the question.
The optional explanation argument is used to give feedback to the user in case there is an incorrect answer given.

For example, the question can be asked in the body of the document (the position of any hint will be different):

answer = sqrt(2)
tol = 1e-3
numericq(answer, tol,
    label=L"What is $\sqrt{2}$?",
	hint="you need to be within 1/1000")

What is $\sqrt{2}$? 🎁

Using Documenter adjustment

Math markup using $\LaTeX$ in Markdown may be done with different delimiters. There are paired dollar signs (or double dollar signs); paired $ and $ (or \[, \]) delimiters; double backticks (which require no escaping); or even math flavors for triple backtick blocks. When displaying $\LaTeX$ in HTML, the paired parentheses are used. However with Documenter paired dollar signs are needed for markup used by QuizQuestions. As of v"0.3.21", placing the line ENV["QQ_LaTeX_dollar_delimiters"] = true in make.jl will instruct that. This package documentation illustrates.

Examples of question types

Choice questions (one from many)

The radioq question was shown above.

The buttonq question is alternative to radio buttons where the correct answer is shown after the first choice. If this choice is wrong, the explanation is shown along with the correct answer.

buttonq([L"1 + 1", L"2+2", L"-1 + -1"], 1;
    label = L"Which adds to $2$?",
	explanation="Add 'em up")

Which adds to $2$?

Multiple choices (one or more from many)

A multiple choice question (one or more from many) can be constructed with multiq:

choices =[
	"Four score and seven years ago",
	"Lorum ipsum",
	"The quick brown fox jumped over the lazy dog",
	"One and one and one makes three"
]
answer = (1, 4)
multiq(choices, answer,
    label="Select the sentences with numbers (one or more)")

Select the sentences with numbers (one or more)

The quick brown fox jumped over the lazy dog

Four score and seven years ago

One and one and one makes three

Lorum ipsum

The multibuttonq question is similar, but it uses a "done" button to initiate the grading. This allows for answers with $0$, $1$, or more correct answers.

choices =[
	"Four score and seven years ago",
	"Lorum ipsum",
	"The quick brown fox jumped over the lazy dog",
	"One and one and one makes three"
]
answer = (1, 4)
multibuttonq(choices, answer,
    label="Select the sentences with numbers (one or more)")

Select the sentences with numbers (one or more)

Numeric answers

Questions with numeric answers use numericq. The question is graded when the input widget loses focus.

answer = 1 + 1
numericq(answer;
    label=L"1 + 1?",
	hint="Do the math")

$1 + 1?$ 🎁

Numeric questions can have an absolute tolerance set to allow for rounding.

Text response

A question graded by a regular expression can be asked with stringq. The question is graded when the input widget loses focus.

stringq(r"^Washington"; label="Who was the first US president?",
        placeholder="last name")

Who was the first US president?

Matching

A question involving matching can be asked with matchq.

questions = ("Select a Volvo", "Select a Mercedes", "Select an Audi")
choices = ("XC90", "A4", "GLE 350", "X1") # may be more than questions
answer = (1,3,2) # indices of choices that match correct answer for each question
matchq(questions, choices, answer;
    label="For each question, select the correct answer.")

For each question, select the correct answer.

Select a Volvo

Select a Mercedes

Select an Audi

The above shows that the number of choices need not match the number of questions. When they do, a dictionary can be used to specify the choices and the answers will be computed:

d = Dict("Select a Volvo" => "XC90", "Select a Mercedes" => "GLE 350",
         "Select an Audi" => "A4")
matchq(d, label="Match the manufacture with a model")

Match the manufacture with a model

Select a Mercedes

Select a Volvo

Select an Audi

Fill-in-the-blank questions

Fill-in-the blank questions can be asked with fillblankq. Answers can be gathered and graded in different manners.

question = "The quick brown fox jumped over the ____ dog"
fillblankq(question, ("lazy", "brown", "sleeping"), 1)

The quick brown fox jumped over the Make a selection dog

(like stringq)

question = "The quick brown fox jumped over the ____ dog"
fillblankq(question, r"^lazy$")

The quick brown fox jumped over the Fill in the blank dog

(like scriptq)

funct = "(input) => input >= 42"
scriptq(funct, label="a large number", explanation="should be at least 6 * 7")

a large number

(like numericq)

question = "____ `` + 2  = 4``"
fillblankq(question, 2)

Fill in the blank $+ 2 = 4$

Select from an image question

The hotspotq shows an image, specified by a file, and grades an answer correct if a mouse click is in a specified rectangular region. The region is given in terms of (xmin,xmax) and (ymin, ymax) as if the entire region was in $[0,1] \times [0,1]$, though the correct_answer argument allows for more complicated regions.

using Plots
p1 = plot(x -> x^2, axis=nothing, legend=false)
p2 = plot(x -> x^3, axis=nothing, legend=false)
p3 = plot(x -> -x^2, axis=nothing, legend=false)
p4 = plot(x -> -x^3, axis=nothing, legend=false)
l = @layout [a b; c d]
p = plot(p1, p2, p3, p4, layout=l)
imgfile = tempname() * ".png"
savefig(p, imgfile)
hotspotq(imgfile, (0,1/2), (0, 1/2),
    label=L"What best matches the graph of $f(x) = -x^4$?")

What best matches the graph of $f(x) = -x^4$?

The PlotlyLight package provides a very lightweight interface for producing JavaScript based graphs with the plotly.js library. The plotlylight type allows questions involving an (x,y) selection from a graph ((x,y) is a point on a graph).

Note

The plotlylight question type does not work with Documenter.

Reference

The available question types are listed below. If others are desirable, open an issue on the GitHub repository.

QuizQuestions.radioq — Function

radioq(choices, answer; label="", hint="", explanation="", keep_order=false)

Multiple choice question (one of several)

Arguments:

choices: indexable collection of choices. As seen in the example, choices can be formatted with markdown.
answer::Int: index of correct choice
keep_order::Boolean: if true keeps display order of choices, otherwise they are shuffled
inline::Bool: hint to render inline (or not) if supported
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection

Example:

choices = ["beta", raw"``\beta``", "`beta`"]
answer = 2
radioq(choices, answer; hint="Which is the Greek symbol?")

source

QuizQuestions.buttonq — Function

buttonq(choices, answer; label="", hint="", [colors])

Use buttons for multiple choice (one of many). Show answer after first click.

Arguments:

choices: indexable collection of choices. As seen in the example, choices can be formatted with markdown.
answer::Int: index of correct choice
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection
colors a named tuple with colors for GREEN, RED, and BLUE

Example:

choices = ["beta", raw"``\beta``", "`beta`"]
answer = 2
explanation = "The other two answers are not a symbol, but a name"
buttonq(choices, answer; label="Which is the Greek symbol?",
        explanation=explanation)

source

QuizQuestions.yesnoq — Function

yesnoq(ans; [label, hint, explanation])

Boolean question with yes or no labels.

Examples:

yesnoq("yes")
yesnoq(true)

source

QuizQuestions.booleanq — Function

booleanq(ans; [label, hint, explanation])

True of false questions:

Example:

booleanq(true; label="Does it hurt...")

source

QuizQuestions.multiq — Function

multiq(choices, answers; label="", hint="", explanation="", keep_order=false)

Multiple choice question (one or more of several)

Arguments:

choices: indexable collection of choices. As seen in the example, choices can be formatted with markdown.
answers::Vector{Int}: index of correct choice(s)
keep_order::Boolean: if true keeps display order of choices, otherwise they are shuffled
inline::Bool: hint to render inline (or not) if supported
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection

Example

choices = ["pear", "tomato", "banana"]
answers = [1,3]
multiq(choices, answers; label="yellow foods", hint="not the red one!")

source

QuizQuestions.multibuttonq — Function

multibuttonq(choices, answers; label="", hint="", explanation="", keep_order=false)

Multiple choice question (zero, one or more of several) using buttons and a "done" button to reveal the answers and whether the user input is correct.

Arguments:

choices: indexable collection of choices. As seen in the example, choices can be formatted with markdown.
answers::Vector{Int}: index of correct choice(s)
keep_order::Boolean: if true keeps display order of choices, otherwise they are shuffled
inline::Bool: hint to render inline (or not) if supported
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection

Example

choices = ["pear", "tomato", "banana"]
answers = [1,3]
multibuttonq(choices, answers; label="yellow foods", hint="not the red one!")

source

QuizQuestions.matchq — Function

matchq(questions, choices, answers; label="", hint="", explanation="")
matchq(d::Dictionary; label="", hint="", explanation="")

Use a drop down to select the right match for each question.

Arguments:

questions: Indexable collection of questions
choices: indexable collection of choices for each question. As seen in the example, choices can be formatted with markdown.
answers: for each question, the index from choices of the correct answer
d: As an alternative, a dictionary of questions and answers can be specified. The choices will be taken from the values then randomized, the answers will be computed
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection

Examples

questions = ("Select a Volvo", "Select a Mercedes", "Select an Audi")
choices = ("XC90", "A4", "GLE 350", "X1") # may be more than questions
answer = (1,3,2) # indices of correct
matchq(questions, choices, answer)

This example uses a dictionary to specify the questions and choices:

d = Dict("Select a Volvo" => "XC90", "Select a Mercedes" => "GLE 250")
matchq(d)

source

QuizQuestions.numericq — Function

numericq(value, atol=1e-3; label="", hint="", units="", explanation="", placeholder=nothing)

Match a numeric answer

Arguments:

value: the numeric answer
atol: $|\mathrm{answer} - \mathrm{value}| \le \mathrm{atol}$ is used to determine correctness
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
units: a string indicating expected units
placeholder: text shown when input widget is initially drawn

source

QuizQuestions.stringq — Function

stringq(re::Regex; filter::Regex=r"", label="", hint="", explanation="", placeholder="")

Match string answer with regular expression

Arguments:

re: a regular expression for grading
filter: a regular expression for what to remove from the string before matching (e.g. r"\s" to remove whitespace)
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection
placeholder: text shown when input widget is initially drawn

Example

re = Regex("^abc")
stringq(re, label="First 3 letters...")

source

QuizQuestions.scriptq — Function

scriptq(funct::AbstractString, runonce::AbstractString=""; label="", hint="", explanation="", placeholder="")

Check string answer with custom script

Arguments:

funct: a string representing the name of a JavaScript function/closure of signature (str: string): boolean
label: optional label for the form element
hint: optional plain-text hint that can be seen on hover
explanation: text to display on a wrong selection
placeholder: text shown when input widget is initially drawn

Example

// in JavaScript
function threshold(t) {
    return (input) => input >= t;
}

# in Julia
stringq("threshold(42)", label="A large number")

source

QuizQuestions.fillblankq — Function

fillblankq(question answer::Regex; placeholder=nothing, [label, hint, explanation])
fillblankq(question, choices, answer; keep_order=false,[label, hint, explanation])
fillblankq(question, val, atol=0; placeholder=nothing, [label, hint, explanation])

Present a fill-in-the-blank question where the blank can be a selection, a number, or a string graded by a regular expression.

question: A string. Use ____ (4 or more under scores) to indicate the blank.

Other rguments from stringq, radioq, and numericq

Examples

question = "The quick brown fox jumped over the ____ dog"
fillblankq(question, r"lazy")
fillblankq(question, ("lazy", "brown", "sleeping"), 1)
fillblankq("____ ``+ 2  = 4``", 2)

source

QuizQuestions.hotspotq — Function

hotspotq(imagefile, xs, ys=(0,1); label="", hint="", explanation="",
    correct_answer=nothing)

Question type to check if user clicks in a specified region of an image.

imgfile: File of an image. The images will be encoded and embedded in the web page.
xs: iterable specifying (xmin, xmax) with 0 <= xmin <= xmax <= 1
ys: iterable specifying (ymin, ymax)
correct_answer: A text snippet of JavaScript which can be specified to add more complicated logic to test if an answer is correct. It must use x and y for the coordinates of the click.

Examples

using Plots
p1 = plot(x -> x^2, axis=nothing, legend=false)
p2 = plot(x -> x^3, axis=nothing, legend=false)
p3 = plot(x -> -x^2, axis=nothing, legend=false)
p4 = plot(x -> -x^3, axis=nothing, legend=false)
l = @layout [a b; c d]
p = plot(p1, p2, p3, p4, layout=l)
imgfile = tempname() * ".png"
savefig(p, imgfile)
hotspotq(imgfile, (0,1/2), (0, 1/2), label="What best matches the graph of ``f(x) = -x^4``?")

Note

The display of the image is not adjusted by this question type and must be managed separately.

source

QuizQuestions.plotlylightq — Function

plotlylightq(p, xs=(-Inf, Inf), ys=(-Inf, Inf);
             label="", hint="", explanation="",
             correct_answer=nothing)

From a plotly graph of a function present a question about the x-y point on a graph shown on hover. (For figures with multiple graphs, the hover of the first one is taken. For parameterized plots, the hover may be computed in an unexpected manner.)

By default, correct answers select a value on the graph with x in the range specified by xs and y in the range specified by ys.

xs: specifies interval for selected point [x₀,x₁], defaults to (-Inf,Inf)
ys: range [y₀,y₁]
correct_answer: When specified, allows more advanced notions of correct. This is a JavaScript code snippet with x and y representing the hovered point on the graph that is highlighted on clicking.

Examples

using PlotlyLight
xs = range(0, 2pi, length=100)
ys = sin.(xs)
p = Plot(Config(x=xs, y=ys))
plotlylightq(p, (3,Inf); label="Click a value with ``x>3``")

An example where the default grading script needs modification. Note also, the x, y values refer to the first graph. (One could modify their definition in correct answer; they are found through x=e.points[0].x, y=e.points[0].y.)

xs = range(0, 2pi, length=100)
ys = sin.(xs)
p = Plot(Config(x=xs, y=ys));
push!(p.data, Config(x=xs, y=cos.(xs)));  # add layer
question = "Click a value where `sin(x)` is increasing"
# evalute pi/2 as no pi in JavaScript, also Inf -> Infinity
correct_answer = "((x >= 0 && x <= 1.5707963267948966) || (x >= 4.71238898038469 && x <= 6.283185307179586))"
plotlylightq(p; label=question, correct_answer=correct_answer)

Note

The use of PlotlyLight graphics works with Weave and Pluto, but is unusable from quarto and Documenter.

source

QuizQuestions.scorecard — Function

scorecard(values; oncompletion::Bool=false, not_completed_msg::String="", which_attempt=:completed)

Add a scorecard

values is a collection of pairs, interval => message. See below. The default summarizes the number of correct/attempted of the number of questions attempted and the total number of questions.
The oncompletion flag can be set to only show the message if all the questions have been attempted. When set, and not all questions have been attempted, the not_completed_msg message is shown.
which_percent is either :correct or :attempted

The interval is specified as (l,r), with 0 <= l < r <= 100, or optionally as (l,r,interval_type) where interval_type, a string, is one of "[]", "[)","(]", or "()". The default interval type is "[)" unless r is 100, in which case it is "[]".

The message is shown when the percent correct/attempted is in the interval. The following values are substituted, when present:

{{:total_questions}} - the number of total questions
{{:correct}} - the number of correct answers
{{:attempted}} - the number of attempted questions
{{:total_attempts}} – the number of attempts.

The message may have Markdown formatting.

Example

vals = [(0,99)=>"Keep trying",
          (99, 100) => "You got {{:correct}} **correct** of {{:total_questions}} total *questions*"]
scorecard(vals)

source