Making an experiment – General steps
Set up an experiment file – makeExperiment
Set trial repetition and order – Trial list
Define objects not specific to trial (if needed)
Making an experiment – Options
Copy all cells in a column – *#
Generate combinations across columns – *
Randomize order of cells in a column – ?
Leave properties at default/unset
Experiment results output – info fields – #
Different trial structures – Multiple sheets
Custom trial definition numbering/names
[Set trial repetition and order in trial definitions – n]
Loading and running an experiment
Load to build in memory – loadExperiment
View in table – viewExperiment
Test using auto response – runExperiment -a
Quit/Resume – Ctrl + Esc, saveExperiment, loadExperiment
Other tools – showElements, recordElements, etc.
Randomize order of combinations of values (exhaustive)
Randomize order of combinations of values (non-exhaustive)
Counterbalance combinations of values
Randomize values independently
Randomize/Counterbalance across subjects
Trial groups (blocks, training trials, etc.)
One-off trials (intro, breaks, etc.)
The visual method works for experiments of low to moderate complexity. The concepts are the same as in the coding method: objects and properties, trial definitions, and the trial list—but we lay it out in tables in a spreadsheet. You can use any Excel-compatible spreadsheet app—Google Sheets is most common since it gives you dropdown menus and quick docs. When done, save/export to an Excel file, use the command loadExperiment in MATLAB to load it, followed by commands like runExperiment, etc. (same as the coding method from that point on).
We’ll look at the general steps first. Then we’ll look at options you can add when needed. See also example experiments.
To start you can use makeExperiment at the MATLAB command line to open a new experiment spreadsheet. This sets you up with tables, formatting, documentation links, etc. If you use Google Sheets, it also includes dropdown menus and quick docs. makeExperiment is optional—you can always just start from a blank spreadsheet in any app.
The main part of making an experiment is making its trial definitions. For each trial this just means setting property values for all the element objects to run in it. You can also set trial object properties if you want to change any away from default (e.g. preTrialInterval). To make trial definitions, use a table in your spreadsheet file with columns = object properties and rows = trial definitions—see example below. You can also use more than one sheet/table if convenient.
The usual approach is to ignore trial repetition and order to run in when defining trials. We’ll add that in separately later (Trial list). Instead just write each distinct trial once, and in any convenient order. This just means each distinct combination of property values across objects. Often you can make all the trial definitions you need in just a few rows.
For what follows see examples below to help visualize...
This is where you specify what objects will be in the trial definitions in the table. Each heading is an object type name (e.g. picture).
Optionally you can add an object variable name (e.g. stimulus), separated by a space or line break. Object variable names can be any valid MATLAB variable names. They appear in places like experiment results output. They are also necessary if you use multiple objects of the same type, to distinguish them. You can add an index (or indexes) after an object variable name if you want to organize multiple objects into the same variable, e.g. stimuli(1), stimuli(2), stimuli(3,4), etc. If you don’t specify an object variable name, default is same as object type name.
This is where you specify names of properties you want to set for the objects in row 1. You can also use expressions to set parts of properties, e.g. end.duration to set field duration of struct property end, position(1) or position(2) to set one component of vector property position, etc. You don't need to include columns for properties you want to leave at default in all trial definitions in the table (usually most properties).
Typically group property columns for an object under the same object heading. You can also set properties for an object in groups of columns that aren't next to each other—just repeat the object heading at each group.
(Google Sheets)
In Google Sheets you can use dropdown menus for object and property headings. However, you can also type over any dropdown to make a custom heading (e.g. add an object variable name in a row 1 heading).
You can optionally add a third heading row that contains options for columns. For example, you can put -s in there for a column to tell MATLAB to read all values in the column as single strings and not need quotes (see below). Heading row 3 is optional and you can omit it or leave it blank if you want.
Each row after the heading rows is a trial definition. In each one set values for the targets you made headings for. These are just MATLAB values in MATLAB syntax, e.g. <cd>2<cd>, <cd>true<cd>, <cds>"hello"<cds>, <cd>[1 2 3]<cd>, <cd>{<cd><cds>'a' 'b' 'c'<cds><cd>}<cd>, etc. More generally they can be any MATLAB expression, e.g. <cd>1/2<cd>, or <cd>randomNum(0, 1)<cd> to set a property to a random number between 0–1. PsychBench evaluates all values/expressions in MATLAB when you load the experiment later.
Single strings – -s/-s"/-s': You can add -s (or -s") in heading row 3 for a column to tell PsychBench to read all values in the column as single <cds>"<cds> strings, or -s' to read as <cds>'<cds> strings. In that case you can omit quotes for all of them. (You cannot do this for arrays of multiple strings, e.g. <cd>[<cd><cds>"hello" "goodbye"<cds><cd>]<cd>.)
Auto copy: If you want the same value/expression for all trial definitions in a column, you can just write it once for the first trial definition. When you load the experiment in MATLAB, PsychBench will automatically copy the cell down for however many trial definitions there are. [Auto copying expressions: In keeping with how spreadsheets work, if you use a MATLAB expression in the cell, PsychBench evaluates each copy independently in its row. For example, if you use an expression like randomNum(0, 1) then each trial definition will get a different random number. As an option you can add $ after an expression to tell PsychBench to evaluate it to a solid value first, then copy that value, e.g. randomNum(0, 1)$.]
Notes about how PsychBench reads spreadsheets:
This table defines three trials corresponding to three different picture files. Each trial definition contains a picture object and a keyPress object. The picture has the default name for its type (picture) and the keyPress has name recorder. For the picture in each trial definition we set properties fileName, height, struct property start field t, struct property end field response, and report. Similarly we set some properties for the keyPress. Each property has the same values in all trial definitions except picture fileName. All properties not listed are left at default. Values are all in MATLAB syntax. The exception is we don't need quotes in column 1 since we use the -s flag to tell PsychBench to read all values in it as single strings.
After you have defined your distinct trials, you can add repetition and order to run in. The usual way is to set a trial list, which is just a list of trial definitions for the experiment to run through. To do this, use a separate sheet/table where you fill in just one cell (you can name the sheet if you like). By default PsychBench automatically numbers trial definitions 1, 2, 3, … in the order they appear across your trial definition tables, and the trial list is just a MATLAB vector with numbers drawn from those. You can repeat numbers in the vector. You can order numbers however you want, including randomizing order in parts or in whole. As usual you can use a MATLAB expression to set the vector—little functions rep and randomOrder in <><PsychBench folder><>/tools are useful here. There are also options you can use for more complex experiments: different trial structures, custom trial definition numbering/names.
Repeat running trials with expressions: The trial list doesn't affect any rows in your tables. It applies during the experiment to run the trial definitions your tables have set. At that point any MATLAB expressions in your trial definitions have been evaluated to solid values. This matters if you use an expression like randomNum(0, 1): if you repeat run that trial definition, each run will have the same random number. If you want different random numbers, make multiple trial definitions, one for each potentially distinct value (see example in Randomization).
The trial list is optional. If you don’t set it, the experiment just runs each trial definition once in number/name order. [Or for unusual cases you can set repetition and order directly in trial definitions.]
This adds a second "table" with just one cell containing a trial list. The three trial definitions have default numbers 1, 2, 3. The experiment will run each of the three trial definitions twice, all in random order. The trial list expression uses functions randomOrder and rep available in <><PsychBench folder><>/tools (you can try the expression at the MATLAB command line).
The first table also splits property report for the two objects into a set of columns together at the end. Note there are still only two objects being set, with all properties the same as before. This is just a change in layout for personal preference.
If you need to set any properties of objects that are not specific to trial—e.g. an experiment object, screen or other device objects, staircase objects, etc.—do this in a separate sheet/table. This is like a trial definition table except it only has two heading rows and one row of values. If you don't need any of those objects, ignore this step.
This table defines an experiment object and a screen object and sets some properties for each.
There are various options you can add to the basic steps above. In any given experiment you won’t use most of these. But you can use any of them as needed:
In addition to standard features in your spreadsheet app (e.g. spreadsheet formulas), there are some PsychBench options you can use to copy and reorder table cells in trial definition tables. PsychBench applies these to your tables when you load the experiment in MATLAB:
You can tell PsychBench to copy (tile) all cells in a column down a certain number of times. To do this, put *# in heading row 3 (options) for the column, where is # number of times to copy, e.g. *5. PsychBench will add rows if needed.
If you want to make a trial definition for each combination of values across two or more columns, optionally you can just write the distinct values/expressions in each column and tell PsychBench to generate combinations. To do this, put * in heading row 3 (options) for each of the columns. PsychBench will replace the columns with one combination per row, adding rows if needed. [For cases where it matters, order of combinations is what would be generated by for loops nested outer to inner in the order * columns appear.]
Treat columns as one column – *[ ]
You can use *[name] for some columns to treat them as a single column—cells across those columns act as one set for combining with other * columns, and they keep lined up in the resulting combinations. Here name is any text label you want to identify the group, e.g. *[b]. Note columns in a [ ] group don’t need to be next to each other or under the same object.
Generate combinations within groups
If you want to generate combinations separately for different sets of columns, you can use *name for each set. Here name is any text label you want to identify a group, e.g. *A. Note you can combine this with [ ] groups, e.g. *A[b].
You can tell PsychBench to randomize the order of cells in a column. To do this, put ? in heading row 3 (options) for the column. (Note this is useful for randomizing order relative to other columns. To randomize the order whole trials run in, you can just randomize their order in the trial list. See Randomization.)
Keep cells across columns lined up – ?[ ]
You can use ?[name] for some columns to give them the same random order so their cells keep lined up. Here name is any text label you want to identify the group, e.g. ?[b]. Note columns in a [ ] group don’t need to be next to each other or under the same object.
If you want the same value/expression for all trial definitions in a column, you can just write it once for the first trial definition. PsychBench will automatically copy the cell down for however many trial definitions there are in the table (after applying any *, *# in other columns—see above). More generally auto copy applies anywhere in a column where you leave cells empty: any empty cell gets a copy of the last non-empty cell above it.
You can tell PsychBench to copy one or more whole columns to apply them to multiple objects, as long as those objects have the the same object type. That way you only have to write the columns once. In the object heading for the columns, after the object type name, list multiple object variables separated by spaces/commas/semicolons/line breaks. e.g. for two picture objects: picture stimulus distractor. And/Or you can list multiple objects in the same variable using indexes to the variable and standard MATLAB syntax: e.g. picture stimuli(1:2). Note you can still use separate object headings for other columns if you want to set some properties differently for the objects (see example below).
Combine options: If you combine more than one of these options for a column, they work in the order listed above. Or if you combine any of *#, *, ?, you can specify the order they work in by the order you list them in heading row 3.
Copying/Reordering expressions: In keeping with how spreadsheets work, if you use a MATLAB expression in cell(s) and then an option that copies or reorders the cells (i.e. any of the options above), PsychBench evaluates each copy independently in its new location. For example, if you use an expression like randomNum(0, 1) then each copy will evaluate to a different random number. Or if you reference another property/variable from earlier in the row, then each copy will reference the value from its row. As an option you can add $ after an expression to tell PsychBench to evaluate it to a solid value first, then copy/move that value, e.g. randomNum(0, 1)$.
This table defines 11 trials, each with a different picture rotation and opacity. Which opacity goes with which rotation is randomized by randomizing order in one of those columns. Each other property has the same value in all trial definitions.
This table copies the first three columns for each of bmlWalker objects stimulus, masks(1), and masks(2). , so each of those objects shares the values sets in those columns. Note fileName is different in the two trial definitions but still the same for the three objects within each trial definition. The table similarly sets scramble the same for masks(1) and masks(2) but different for stimulus.
If you use a MATLAB expression in a table cell, you can reference other object properties in the row, make and reference "free variables", and reference or fill from external variables.
You can make property values dependent on other properties. To do this, reference object variable + property in the expression you set, e.g. stimulus.color, stimuli(2).color, etc. The referenced property must be in an earlier column and in the same row.
You can also make property values dependent on "free variables" which you set with any names and values. To define a variable, add a column anywhere before the first object in the table, leave heading row 1 (object heading) blank, and specify a variable name in heading row 2. Like object variable names, these can be any valid MATLAB variable names. Set values in the column just like you would for a property. You can then reference the variable by name in expressions in later columns in the same row.
Defining free variables works the same as setting properties. For example, you can use separate column headings to set parts of a variable, e.g. var(1), var(2) to set the numbers in a 1×2 vector variable separately. You can also use column options for them.
You can also reference external variables which you will set values for later in MATLAB when you load the experiment. This lets you modulate an experiment at each run. Provide these variables as additional inputs to loadExperiment. In expressions in your tables reference them by the names they will have in the workspace where you call loadExperiment and that you will input them with. The order you input them to loadExperiment doesn't matter, only their names. You can use external variables anywhere, even in the trial list.
Fill cells in a column from a vector/array – ^: If your external variable is a vector/array, instead of referencing the whole variable in one cell, you can tell PsychBench to fill its values into cells in a column. To do this put ^ before the variable name or expression, e.g. ^myArray. PsychBench will fill starting at the cell containing the ^ and into any empty cells below it, adding rows after if needed. [If filling from a row/column array, values draw from across its length. If filling from a multi-dimensional array, values draw from across its rows (dimension 1). If filling from a row/column cell array, values are extracted from its cells for convenience.]
Note references are in MATLAB expressions, so they refer to solid MATLAB values. For example, if you use an expression like randomNum(0, 1) to set a property and then reference that property in an expression later in the same row, it will take the same random number.
In this table we make a variable called path with the same value in every trial definition, and reference it in expressions setting picture property fileName. We also reference what we set in picture property info field n_picture when setting text property text. We reference the picture object by its name pic, specified in its heading. By default each copy of the pic.info.n_picture expression is evaluated independently in its row, so text.text will have values 1, 2, 3.
You don't need to include columns for properties you want to leave at default (unset) in all rows in a table, which is usually most properties. Aside from that, in trial definition tables there are a few further options for unusual needs:
→ Set the property = <cd>[]<cd> (MATLAB empty).
→ More generally you can put - in a cell to mean don’t set the property/target at all.
→ Make one column for the object and set the object heading (row 1) but leave the property heading (row 2) blank. This makes the object in each trial definition but doesn’t set any properties, so all are left at default. (Additionally you can put - in cells in the column where you don’t want to make the object at all in those trial definitions.)
→ If you set every cell in a row = -, the trial is still defined (gets a trial definition number) but will skip running wherever it is used in the experiment because no objects are set in it.
For each object, you can specify what to see in experiment results output (if anything) using properties info and report. info is a struct where you can add any fields you want containing any values you want. This doesn't affect how the object works—it just adds custom information to see in results. You can set info fields normally using headings info.<field name> in heading row 2. However, as a shortcut you can also just write #<field name>:
This shortcut syntax is only for info fields. If you need report, set it normally as a property.
The following two tables are equivalent. The columns are part of four trial definitions, each showing two rectangles. In experiment results output we want to see a condition number we make listed under trial n_condition, a direction label we make listed under direction for each rectangle, and the values of properties rotation and duration for each rectangle. (Only columns relating to results output are shown.)
The trials (rows) × properties (columns) layout of a trial definition table is good for the simple case where every trial has the same structure, i.e. similar objects with just a few property values differing. If you need some trials with very different structures—e.g. intros, breaks, outros, syncs, training trials, etc.—you can use multiple trial definition sheets/tables with different heading rows. PsychBench just continues numbering trial definitions 1, 2, 3, … across tables for use in the trial list. You can also mix in custom numbering/names...
You can give some trial definitions custom numbering or names. This can be important in more complex experiments in order to have trial definition identifiers that are easy to work with in the trial list—for example, giving trial definitions that will run together adjacent numbers so you can use simple vector syntax like <cd>101:120<cd>. To do this, add a column with no headings before the first object in a table (optionally you can give it the heading n_def.) You can use options for it just like for property columns. There are two kinds of values you can put in this column:
PsychBench numbers trial definitions that have the same base number n as n+1, n+2, n+3, … in the order they appear across tables. For example, you can use 100 in a set of trials to tell PsychBench to number them 101, 102, 103, … (Default = 0 → 1, 2, 3, …). Note you can use auto copy on a group number column, like any other column.
PsychBench simply names the trial definition with the string. Useful for one-off "trials" like intros, breaks, sync with scanner, etc. Then in the trial list you can mix numbers and names using a cell array of vectors and strings, e.g. <cd>{<cd><cds>"intro"<cds><cd> 1:8 <cd><cds>"break"<cds><cd> 1:8}<cd>. Note you can repeat names in the trial list just like numbers.
This example adds intro and outro “trials” to the basic example we started with above. It adds these two trial definitions in a second table since they have a different structure from the task trials, and then adds them to the trial list to run at start and end of the experiment. See also for example walkerDirectionsDemo which includes training trials using a group number.
For unusual cases you can set trial repetition and order to run in directly in trial definitions. In that case don't set a trial list. In each trial definition table add a column anywhere before the first object and give it heading n. Then for each trial definition set a trial number to run as during the experiment (1, 2, 3, … in any order). If you want to repeat a trial definition, you can give it a vector of numbers. You can use options for it just like for property columns. Usually this is harder than using a trial list but the option is available.
You can tell PsychBench to ignore any column in a table by putting % in heading row 1 for it, before the heading if any. You can also tell it to ignore a whole sheet by putting % before its name. (Like commenting out in MATLAB.)
When you have your experiment design made, save/export it to an Excel file (in Google Sheets: File → Download → Microsoft Excel), Then use the following commands in MATLAB to load and run it. As usual type help <><function name><> in MATLAB for usage of each.
Use loadExperiment to load an Excel experiment file into MATLAB. This evaluates all MATLAB values/expressions in the spreadsheet and builds the experiment in memory. Generally use loadExperiment once for each run/subject to redo any randomization in the spreadsheet (e.g. expressions using randomNum or randomOrder).
Once you have loaded an experiment, everything else is the same as for the coding method...
Use runExperiment to run the experiment that is in memory. runExperiment outputs results in variables, and optionally saves them to .mat and .csv files.
Aside from these basics steps, there are options you can add in as needed:
Before running an experiment for the first time, it can be helpful to check it in table form in MATLAB. After using loadExperiment to build it in memory, use viewExperiment at the MATLAB command line to see the trials that will run in table form (in MATLAB). You can also call viewExperiment -d to sort them by trial definition.
Another useful testing feature is running an experiment in auto response mode to simulate a subject. To do this call runExperiment -a. You can change auto response options in properties autoResponse, autoResponseLatency which all response handler elements have.
You can press Ctrl + Esc to quit running an experiment part way through.
If you quit an experiment part way through and then call runExperiment again, it asks if you want to resume the experiment at the trial you quit. Results output and time measurements will be as if the experiment was never quit. You can only resume an experiment if it’s still in memory, i.e. up until the next loadExperiment, newExperiment, clearExperiment, clear all, or quitting MATLAB. However, most commonly you’ll want to resume in a later session. To do this, use saveExperiment to save the state of the experiment to a .mat file (not an Excel file), then loadExperiment to load it back into memory later.
A few other tools can be used at the MATLAB command line for various tasks. For example, showElements shows/runs element objects without needing to make an experiment, recordElements captures element objects to image or movie files, etc. Also don't forget pb, pb_prefs, pb_update, etc. These are all in the root PsychBench folder—type help PsychBench for an overview.
Below we look at some basic cases of how to randomize conditions across trials. Note you can combine any number of these. However, for complex randomization/balancing the coding method may be necessary.
Remember the general rule: Each trial definition is a distinct trial, i.e. a distinct combination of property values across objects. The trial list then sets repetition and order to run those distinct trials in during the experiment. Little functions in <><PsychBench folder><>/tools are essential here, e.g. randomNum, randomOrder, etc. Type help tools for an overview.
The simplest case is sampling values from a discrete set (factor) a certain number of times each. Generally the way to do this is to make one trial definition for each distinct value, then in the trial list apply repetition (if any) and randomization to the order the distinct trials will run in (use tools rep, randomOrder in the trial list expression). See Example 1 above.
As above except here each trial definition is a combination of values from multiple sets (factors). Again make one trial definition for each distinct combination of values, then in the trial list apply repetition (if any) and randomization of order.
Here we test combinations of values from multiple sets (factors), but not every combination. So, we need to additionally randomize which values appear together. In the most basic case we have sets of equal sizes and we just want to shuffle the correspondence between them. We can do this by writing out the values in each set under the appropriate object+property headings, then use the ? column option to randomize the order in one or more of those columns relative to others. Then still apply randomization to order in the trial list for overall order of presentation. More complex cases may require the coding method. See ? example above.
Like above except when you randomize which values appear together, it is with the constraint that each distinct value in a set appears an equal number of times with each distinct value in another set (and vice versa). Currently this is best done with the coding method. An option for counterbalancing may be added to the visual method later (email contact@psychbench.org).
Another basic case is independent random sampling—for example, when each trial gets an independent random number from a given range. This often involves using a tool like randomNum or randomNum_normal to set the relevant object property value in each trial. The thing to note is that each trial gets a potentially different random value, so each trial is a distinct trial and needs its own trial definition (repeating a trial definition in the trial list would just repeat the same random value during the experiment). In the visual method the *# column option is often convenient here—see example below. Since this kind of randomization is in the trial definitions themselves, you don't need to randomize order in the trial list for it. You may still need to randomize order there if you combine with other cases above.
This example runs 50 trials: 25 showing one picture and 25 showing another. In each trial the picture is shown at a different random rotation independently drawn from −90 ... +90 deg. We do this directly in the trial definitions using function randomNum to set picture property rotation. We need a potentially different random value for each of the 50 trials that will run, so we make 50 trial definitions, one for each. Just to avoid writing the expression 50 times, we use *# to copy cells down—× 25 for the column containing two values = 50, × 50 for the column containing the one randomNum expression. (Since the first column now generates 50 rows and the randomNum column has just the one cell, we could also let it auto copy without using *# for it. Either way the expression re-evaluates at each copy by default.) We randomize order in the trial list just to randomize the appearance of the two pictures. See also walkerDirectionsDemo for another example.
Here different subjects (runs of the experiment) receive different values taken from a larger set, typically so you can counterbalance values across subjects. To do this, in MATLAB create arrays of values at the across-subjects level as needed, use tools like randomOrder or randomBalance to randomize them, and save the arrays to a .mat file. Then at each run of the experiment, load the file, draw the values for that run, and apply them as inputs to loadExperiment. In your experiment spreadsheet reference them as external variables where needed. More complex cases may require the coding method.
Making groups of trials to run in the experiment—such as blocks, or a set of training trials to start the experiment with—is nothing special. Just specify those trials to run together where you want in the trial list. However, often it's convenient to give those trial definitions a custom range of adjacent numbers so you can use simple vector syntax like <cd>101:120<cd> in the trial list.
Special events like an intro and breaks are usually best implemented as one-off trials. Like above, just specify them to run where you want in the trial list. Often it's convenient to give one-off trials custom names as opposed to numbers, just for clarity. The standard experiment template that you get if you use makeExperiment includes a sheet with some typical one-off trials pre-made in it. If you want to use them, remove the % before the sheet name and edit as needed.