Building and running an experiment

In PsychBench we build an experiment by writing an experiment script. This isn’t really code in the usual sense—more like setting a list of parameters. An experiment script builds an experiment in memory. Follow with the command runExperiment (or include it in the script) to run the experiment. Below we’ll outline the main steps, then follow with some examples for experiment designs.

Start building – newExperiment

Call newExperiment at the start of every experiment script. This tells PsychBench to set up for building an experiment in memory, including clearing any experiment currently in memory.

Define distinct trials in any order – addTrial

The main part of writing an experiment script is defining trials. For each trial this just means making all the element objects that can run in it. You can also make an optional trial object, e.g. if you want to change a trial property like preTrialInterval. When you have all the objects for a trial, input them to addTrial to define the trial in memory, where runExperiment will find it later.

The usual approach is to ignore trial repetition and order when defining trials. Instead define each distinct trial once, meaning each distinct combination of property values across all objects in a trial. Typically this is equivalent to each condition or combination of conditions you will test. Little tools like randomNum, randomBalance, etc. in <PsychBench folder>/tools can be useful here. You can define distinct trials in any convenient order, so you can use for loops to automate defining them. We’ll see what this looks like in example experiment designs below...

Set trial repetition and order – setTrialList

After you have defined all distinct trials, it’s time to add trial repetition and order. Do this by calling setTrialList and inputting a list of trial definition numbers/labels for the experiment to run through. By default PsychBench numbers trial definitions 1, 2, 3, ... as addTrial is called. You can also use custom numbers or strings—see addTrial function help and the Special “trials” example below.

You can repeat trial definitions in the list. You can also randomize the order of trials in parts or whole. You can run blocks by grouping in the list. Little tools replicate, randomOrder, randomBalancePerms, etc. in <PsychBench folder>/tools can be useful here.

setTrialList is a common step but optional. If you don't call it, the experiment will just run through the distinct trials in order with no repetition. Alternatively in unusual cases you can set repetition and order in the trial definitions themselves using trial object property n.

Add objects not specific to trial (optional) – addToExperiment

You might need optional objects that are not specific to trial, e.g. an experiment object, screen or other device objects, and staircase objects. Make these objects like element objects using <type name>Object, e.g. screenObject. Then add them to the experiment in memory using addToExperiment.

Check before running (optional) – viewExperiment

Before running an experiment for the first time, you can view it in table form to check that your script set everything as expected. After running an experiment script to build an experiment in memory, you can call viewExperiment at the MATLAB command line to see the trials that will run, and/or viewExperiment -d to see just your trial definitions.

Run – runExperiment

Finally follow with the function runExperiment to run the experiment that you’ve built in memory. You can call runExperiment from the MATLAB command line or just include it at the end of the experiment script.

Save and run for multiple subjects

Saving an experiment design to run for multiple subjects usually just means saving the experiment script. After running the script to build the experiment in memory, you can re-call runExperiment any number of times to re-run it. However, usually you’ll re-run the whole experiment script to re-build for each subject so that any randomization is refreshed.

In unusual cases you can use saveExperiment after building an experiment in memory to save what is actually in memory to a .mat file. Usually you don’t need to do this.

(Quit – Ctrl + Esc)

You can press Ctrl + Esc to quit an experiment before it ends.

(Save and resume – saveExperiment, loadExperiment)

If you quit an experiment part way through and then call runExperiment again without clearing the experiment from memory, it will give the option to resume the experiment at the trial you quit on. Type help runExperiment for details. You can only resume an experiment if it’s in memory, i.e. up until you call newExperiment, clearExperiment, clear all, or quit MATLAB. Of course, most commonly you’ll want to resume an experiment across different sessions. To do this, use saveExperiment to save the state of the experiment to a .mat file, then loadExperiment to load it back into memory later.

(Clear – clearExperiment)

Calling newExperiment at the start of an experiment script clears any experiment currently in memory. clear all or just quitting MATLAB also does it. If for some reason you need a command to clear an experiment and nothing else, you can type clearExperiment.

(See elements outside an experiment – showElements)

You can use showElements if you want to quickly see (run) one or more element objects without needing to build an experiment. You can just make element objects at the MATLAB command line and input them to showElements (no experiment script or addTrial needed). Useful for quickly checking what an element looks like or what effect a property has.

Experiment design examples

One factor sampled from a discrete set

Perhaps the most basic case is testing conditions sampled from a discrete set, with each condition tested an equal number of times and order randomized:

The experiment script below defines 4 distinct trials. Each trial shows a different animal (picture property fileName). The script then runs 3 repetitions of each distinct trial. Randomization is implemented by running the trials in random order. You would re-run the script to re-randomize for each subject.
newExperiment

fileNames = [<cdsm>"dog.jpg" "cat.jpg" "bird.jpg" "fish.jpg"<cdsm>];
<cdkm>for<cdkm> a = 1:4
    pic = pictureObject;
    pic.fileName = fileNames(a);
    pic.start.t = 0;
    pic.end.duration = 0.5;

    <rm><Make other objects for trial><rm>

    addTrial(pic, <rm><other objects><rm>);
<cdkm>end<cdkm>

nn = randomOrder(replicate(1:4, 3));
setTrialList(nn)

[results, resultsMatrix] = runExperiment;
One factor sampled from a continuous distribution

A similar design is testing conditions sampled from a continuous distribution:

In the experiment below, each trial shows the same picture rotated randomly between −30 … +30 deg. This results in a different value for the stimulus property in each trial, so we define each one as its own distinct trial, with 100 trials in total. Since randomization is implemented directly in the elements, we don’t need to do any trial repetition or ordering. Again you would re-run the script to re-randomize for each subject.
newExperiment

<cdkm>for<cdkm> r = 1:100
    pic = pictureObject;
    pic.fileName = <cdsm>"dog.jpg"<cdsm>;
    pic.rotation = randomNum(-30, 30);
    pic.start.t = 0;
    pic.end.duration = 0.5;

    <rm><Make other objects for trial><rm>

    addTrial(pic, <rm><other objects><rm>);
<cdkm>end<cdkm>

[results, resultsMatrix] = runExperiment;
Multiple factors

Testing joint conditions from multiple factors is just an extension of above. Instead of one for loop, use any number of nested for loops. Then each time the body in the loops runs is for a combination of loop variables corresponding to a combination of factors:

Multiple factors sampled from discrete sets

The script below defines 4 × 3 = 12 distinct trials, one for each combination of 4 animals (file names) and 3 opacities. It then runs 3 repetitions of each distinct trial for a total of 36 trials in random order.

newExperiment

fileNames = [<cdsm>"dog.jpg" "cat.jpg" "bird.jpg" "fish.jpg"<cdsm>];
opacities = [0.5 0.75 1];
<cdkm>for<cdkm> a = 1:4
    <cdkm>for<cdkm> o = 1:3
        pic = pictureObject;
        pic.fileName = fileNames(a);
        pic.opacity = opacities(o);
        pic.start.t = 0;
        pic.end.duration = 0.5;

        <rm><Make other objects for trial><rm>

        addTrial(pic, <rm><other objects><rm>);
    <cdkm>end<cdkm>
<cdkm>end<cdkm>

nn = randomOrder(replicate(1:12, 3));
setTrialList(nn)

[results, resultsMatrix] = runExperiment;

Multiple factors sampled from discrete sets / continuous distributions

Below we combine factors sampled from a discrete set (4 animals) and a continuous distribution (50 random rotations) for 4 × 50 = 200 trials in random order. Randomization is implemented in the elements for the continuous factor and in the trial list order for the discrete factor.

newExperiment

fileNames = [<cdsm>"dog.jpg" "cat.jpg" "bird.jpg" "fish.jpg"<cdsm>];
<cdkm>for<cdkm> a = 1:4
    <cdkm>for<cdkm> r = 1:50
        pic = pictureObject;
        pic.fileName = fileNames(a);
        pic.rotation = randomNum(-30, 30);
        pic.start.t = 0;
        pic.end.duration = 0.5;

        <rm><Make other objects for trial><rm>

        addTrial(pic, <rm><other objects><rm>);
    <cdkm>end<cdkm>
<cdkm>end<cdkm>

nn = randomOrder(1:200);
setTrialList(nn);

[results, resultsMatrix] = runExperiment;
Counterbalanced factors

Counterbalancing can mean different things, but here is a common example. Generally the tools randomBalance and randomBalancePerms are useful for counterbalancing.

Below we take the previous example (4 animals × 50 random rotations = 200 trials in random order) and add in randomly counterbalancing two position values: 4 deg left and 4 deg right of center. To do this we use the tool randomBalance to randomly and evenly distribute the two positions across 200 instances (trial definitions) and apply them it with the help of a third index variable that we increment manually:
newExperiment

positions = randomBalance([
    -4  0
    4  0
    ], 200);
fileNames = [<cdsm>"dog.jpg" "cat.jpg" "bird.jpg" "fish.jpg"<cdsm>];

        n_trialDef = 0;
<cdkm>for<cdkm> a = 1:4
    <cdkm>for<cdkm> r = 1:50
        n_trialDef = n_trialDef+1;

        pic = pictureObject;
        pic.fileName = fileNames(a);
        pic.rotation = randomNum(-30, 30);
        pic.position = positions(n_trialDef,:);
        pic.start.t = 0;
        pic.end.duration = 0.5;

        <rm><Make other objects for trial><rm>

        addTrial(pic, <rm><other objects><rm>);
    <cdkm>end<cdkm>
<cdkm>end<cdkm>

nn = randomOrder(1:200);
setTrialList(nn);

[results, resultsMatrix] = runExperiment;
Custom trial numbers, One-off trials (intros, breaks, ...)

You can group trial definitions by telling addTrial to number them from any base number, which can be useful in complex experiments. You can also use a string label for any trial definition instead, which is useful for one-off “trials” like intros, breaks, etc. Type help addTrial for details.

The script below defines two groups of eight distinct test trials each. The first group is numbered 
1–8 (default numbering), the second group 51–58 (base 50 +1, +2, +3, ...). It also defines an intro trial and a break trial with string labels. The experiment runs the intro, the first group of test trials in random order, the break, then the second group in random order. Note the flexible syntax allowing mixed numbers and strings in the input to setTrialList. For another example of complex trials see walkerDirectionsDemo.m.
newExperiment

<cdkm>for<cdkm> n = 1:8
    <rm><Make objects for trial><rm>
    addTrial(<rm><objects><rm>, <rm><objects><rm>, ...);
<cdkm>end<cdkm>

<cdkm>for<cdkm> n = 1:8
    <rm><Make objects for trial><rm>
    addTrial(<rm><objects><rm>, <rm><objects><rm>, ..., 50);
<cdkm>end<cdkm>

<rm><Make objects for intro trial><rm>
addTrial(<rm><objects><rm>, <rm><objects><rm>, ..., <cdsm>"intro"<cdsm>);

<rm><Make objects for break trial><rm>
addTrial(<rm><objects><rm>, <rm><objects><rm>, ..., <cdsm>"break"<cdsm>);

nn = [<cdsm>"intro"<cdsm> randomOrder(1:8) <cdsm>"break"<cdsm> randomOrder(51:58)];
setTrialList(nn);

[results, resultsMatrix] = runExperiment;
Other designs

In real life any complex design is possible (see <PsychBench folder>/docs/demos for a few examples). This makes the full flexibility of MATLAB in experiment scripts essential. The above examples are often more like building blocks you can apply than finished templates. However, the general method is always the same: define distinct trials in any order using for loops, set repetition and order using setTrialList, check using viewExperiment, run using runExperiment, and re-run the experiment script to re-randomize for each subject.