The MatOFF Makdat Command
Introduction
MatOFFuses its own internal file structure to make searches efficient. TheMakdatcommand processes
Cortex files for use withMatOFF.The Makdatcommand not only converts Cortex files to theMatOFFinternal format,
but it also concatenates Cortex files and has powerful filtering features.
Using Makdat (overview)
Before using theMakdatcommand you must first create a "list file". The list file is a list of Cortex data files that are
associated together under one name, called the unit name. In neurophysiological studies, a unit is one neuron recorded from a single spike
channel.Makdatcollects all the data for a single neuron together. A typical list file looks like this:
UNIT br104 CHAN 3 <-- define the name of a unit and all channel number to process with this unit
br104.1 <--- List of Cortex files to include in this unit
br104.2
Once you have a list file (call it "list.lst") it can be used to create a group of files for MatOFFto analyze. (The group name has no file extension.)The list file should be placed in the same directory as the cortex data files. Let's use the group name "br104":
makdat list.lst br104
A display will show the progress. The br104 files will be created. Once theMakdatcommand is done, the new file is ready for analysis with MatOFF. Usually, the first step is to open the new file with theFilecommand.If you don't need any of the other features of theMakdat command, you can just start usingMatOFF. SeeGetting Started with MatOFF. Makdat Features
Makdathas a rich array of features to help with batch process, debugging, data filtering, and file organization. The features are controlled by a group of files and options:
| Makdatfile or option | Purpose |
| List file | This file identifies single units and lists which data files to include. It can also control data filtering. |
| Command options | TheMakdatcommand itself has filtering and debugging options |
| Environmental variables | There are severalenvironmental variablesassociated with theMakdatcommand. |
| Event code file | This file assigns a name for each numeric event code. The file is not essential, but name assignments can aid in debugging. |
| Insert file | Insert is a filtering method for adding codes to the data file. |
| REMAP file | This simple file determines what Cortex event codes are assigned to each spike (pulse) channel number. |
Understanding the basic list file
Thelist fileindicates which Cortex files and spike channels will be included for analysis, and assigns aunitname to that block of data. The list file is an ASCII file created using any standard editor. It begins with the keyword "UNIT" in allcapital letters, and an arbitrary name for the unit. Following the unit name is the keyword CHANNEL (which also must be incapital letters) and a channel number. MatOFF uses the terminology "pulse channel" to identify a discriminated spike. (Other systems use "channel" to mean all the spikes from one electrode.) The next lines of the file are a list of Cortex file names that will be used as data for the unit. To start a new unit, use the UNIT keyword again.
Here is an example list file showing how multiple groups of Cortex files can be assigned to units:
UNIT br35a CHAN 0 <- unit with data only on channel 0 Note: "unit" and "chan" must be capitalized
br35.0 <---
br35.1 | List of Cortex files
br35.2 <---
UNIT br35b CHAN 2-4 <- new unit with data on multiple channels (see below)
br35.0 <---
br35.1 | note overlap of Cortex files with the previous unit
br35.2 |
br35.3 <---
UNIT br36a CHAN 1 <-- beginning of third unit
br36.1 <--- List of Cortex files
br36.2 <---
No extension is assumed for the list file, however the most commonly used extension is ".lst".Note theMatOFFstarts
numbering channels at channel 0, however, a default channel number of 1 will be assigned to UNIT entries that exclude the CHANNEL keyword.
A unit can have more than one pulse channel, but there is no disadvantage to providing a different unit name for data from each pulse channel.
MatOFFis very efficient with data files. If the same data file name shows up repeatedly in a list file, the data are still only copied once.
Command line and command line options
TheMakdatcommand looks like this, note the dash ("-") in front of the option list.
Makdat <list file> <output file>-<option list>
Makdatlooks for the list file using the DATAPATH environmental variable. The <list file> can have any name with any extension.
The <output file> must have no extension. The output file is the root file name for the files generated byMakdat.
Note that the <option list> is preceded by a dash. All Cortex file names listed in the list file must be on the same directory as the <list file>.
TheMakdatcommand can be run with no options as long as both a <list file> and <output file> are specified.
It is possible to run without an <output file> if the -N option is used (see below). It is possible to choose the output file name from within the list file.
Finally, it is possible to establish all the command line options using the MAKDAT environmental variable.
Processing options
| 8 | Default to 8 spike channels using event codes 1 to 8. NormallyMakdatwill default to 2 spike channels using event codes 1 and 2. The default is used only when no spikeREMAPfile is available. |
| N | Do not make an output file. This will override any output file specification on thecommand lineor in thelist file. |
| O | Allow any existing output file to be overwritten. |
| P | Exclude EPP analog data from the data file. Does not exclude EOG data. |
| R | Exclude EOG analog data from the data file. Does not exclude EPP data. |
| T | Ignore Cortex condition codes. Normally Cortex condition codes are automatically translated toMatOFFevent codes by adding the value 200 to the condition code. This option inhibits this translation and makes event codes at 200 and above available for other uses. |
| X | Exclude both EOG and EPP data from the data file. This is the same as specifying options P and R together. |
Debugging options
| A | Show each analog value during processing. |
| C | Treat upper and lower case letters as different letters when reading input file names. (Case sensitive.) |
| D | Show the progress of data processing. |
| E | Inhibit the display of any warning messages. Errors are still displayed. |
| F | List each event code as it is encountered. |
| H | Display the Cortex header at the beginning of each trial. |
| S | Show time and channel of every spike (pulse). |
| V | Verbose mode. List all event codes and number of spikes encountered during processing. |
Option usage and examples
Multiple options are listed together without spaces, e.g. -DMH
| makdat br24.list br24 -x | The list file described above is named br24.list. The "x" option is used to exclude all analog data from the output files. |
| makdat br24.l br24 -o | This command repeats the previous conversion, but includes analog data. Note the "o" option to overwrite the files created previously. |
Makdat environmental variables
TheMakdatcommand uses severalenvironmental variables. The environmental variables can be set in DOS before
running MATLAB or they can be set by theMatOFFSetenvcommand.
| set DATAPATH= <path> | This sets the directory for finding Cortex data files, the List file, theMatOFFoutput files, and the MAKDUMP file. |
| set EOGSAMPLERATE= value | Sample rate of EOG data. The sample rate must be explicitly set either via this variable, or in the list file. The valid values for EOGSAMPLERATE are the same as the PARAM EOGlist file parameter. |
| set EPPSAMPLERATE= value | Sample rate of EPP data. (Not yet implemented.) At the moment, EPP sample rate must match the EOG sample rate. |
| set EVENTCODEFILE= <file name> | <file name> is the file name ofevent codes.MatOFFdoes not look at the DATAPATH variable to locate the Event Code file. The Event Code file specification must include a path. Using the current directory (.\) is allowed. |
| set MAKDAT= <string> | <string> can have any of the standardcommand line options. |
| set MAKDUMP= <file name> | If this variable is defined, allMakdatdebugging information will be sent to this file rather than to the user's screen. |
| set REMAPFILE= <file name> | <file name> is aRemap filefor assigning spike channels to Cortex event codes.MatOFFdoes not look at the DATAPATH variable to locate the Remap file. The Remap file specification must include a path. Using the current directory (.\) is allowed. |
| set MAXINPUTTRIALS= value | Limits the number of trials to analyze. A value of 10 will forceMakdatto process only the first 10 trials of each file. It is usually reserved for debugging purposes. |
| set MAXTRIALSFOUND=value | Limits the number of trials that will be matched to a sequence, and thus the number of trials plotted. Default is 250 and must above that value the array sizes start getting large enough to slow down data processing. |
| set SPIKETIMEOFFSET=value | Adds this amount of time (in milliseconds) to every spike time. It compensates for delays or other offsets built into the spike sorting hardware or software. Absolute values greater than 20 ms are flagged with a warning. |
Event code files
The oF command line option allows you to list the event codes in a Cortex data file. If an event code files is specified, the listing
can include event code names that match your experiment. To supply your own meanings to event codes, create a file with the event code
followed by one space followed by a 1 to 26 character description of the event.
Spaces are allowed within the event description. Example entries:
0 NOCODE
1 SPIKE1
2 SPIKE2
3 RELEASE OF HANDLE
For more examples see the default event code file, DEFAULT.EVC, or the example file, BARNEY.EVC.
It is easiest to create your own .EVC file by modifying the file ALLEVC.EVC.
Once you create the event code file, use theSetenvcommand to make it active:
setenv EVENTCODEFILE=BARNEY.EVC
Dump files
If the MAKDUMP environmental variable is set to a valid file name, information normally sent to the user's screen is
re-routed to this "dump file". A dump file is most useful when using the debuggingcommand line optionslike -F.
Similar data can also be dumped after conversion by the MatOFF commandsDumpandSet.
TheMakdatcommand handles advanced features in the list files. The features include:
- Control of the output files within list files, so one list file can produce multiple output files.
- Setting and changing the A/D conversion rate within the list file.
- Control of inclusion or exclusion of EOG and EPP analog data during processing.
- System for copying event codes from the early part of a trial to later parts of the same trial.
List file entry types
Advanced list files can have three types of entries: unit, file, and parameter. Unit entries are extensions of the simple list file structure.
Most parameter entries are duplicate command line or environmental variable options. The one exception is the "insert" parameter, which controls the
new feature ofinserting codesinto the data stream. The entries are:
| UNIT <unitname> CHANNEL <chan list> | The UNIT keyword establishes the unit name for a group of data files. The unit name can be up to 12 characters in length. The CHANNEL keyword precedes a list of spike (pulse) channels to process. These channel numbers areMatOFFchannel numbers (0 to 99). The lowest channel number in this list is the default channel number for this unit. . The word CHANNEL can be abbreviated as CHAN. Channel list example: 3-7,21,23. Makdat will default to channel 1 if no channel list is given. |
| <file> | <file> is the Cortex file to be processed. It will be converted and stored in the native MatOFFformat. |
| PARAM <parameter> <value> | Parameters: EOG, EPP, XEOG, XEPP, INSERT, XGAIN, YGAIN, REMAP. The word PARAM and the parametersmust be in capital letters. |
| OUTPUT <file name> | Change the specification of the file name to create. The word OUTPUTmust be in capital letters. The file name has no extension, since a group of files will be created based on this name. |
Parameters
The PARAM keyword controls a number of functions:
| EOG | Set the A/D sample rate for EOG data. This parameter can have one of three types of values:
|
| EPP | Not yet implemented. When it is implemented, it will allow a separate A/D rate for EPP data. Presently, EOG data rate must match EOG data rate. |
| XEOG | Exclude EOG channels. (Similar to -Rcommand line option.). XEOG by itself means exclude EOG data from subsequent storage in the output file. XEOG OFF or XEOG FALSE will return EOG to normal processing. |
| XEPP | Exclude EPP channels. (Similar to -Pcommand line option.). XEPP by itself means exclude EPP data from subsequent storage in the output file. XEPP OFF or XEPP FALSE will return EPP to normal processing. |
| XGAIN | Set X gain of EOG data. This becomes a multiplication factor for EOG data. XGAIN stays in effect for all units in file until the command is issued again. An XGAIN of 1.0 returns X channel EOG processing back to normal. Fractional values (e.g., 0.5) are allowed. |
| YGAIN | Set Y gain of EOG data. This becomes a multiplication factor for EOG data. YGAIN stays in effect for all units in file until the command is issued again. An YGAIN of 1.0 returns Y channel EOG processing back to normal. Fractional values (e.g., 0.5) are allowed. |
| IGNORE | Completely ignore a list of codes prior to any processing. |
| INSERT |
Allow codes to be duplicated within a trial. There are two possible parameters: OFF Stop inserting codes. |
| REMAP | Choose aRemap filefor mapping Cortex codes toMatOFFspike channels. |
| SPIKETIMEOFFSET | Add this amount of time (in milliseconds) to every spike time. It compensates for delays or other offsets built into the spike sorting hardware or software. Absolute values greater than 20 ms are flagged with a warning |
| TAG | Insert an event code (tag code) into the start of every trial of this data file. The data file can be processed in blocks, so each block has a different tag code. |
| TRIM | Remove every occurrence of an event code after the first one in each trial. Only one trim code is active at any time. |
| THIN | Do not keep every occurrence of an event code. Keep every 2nd, 3rd, 4th, etc. |
Tag codes
A code can be added to the start of every trial. To add code 6000 to the start of all trials, use:
PARAM TAG 6000
Input files can be broken into blocks, with a different tag code for each block. For example:
PARAM TAG 6001,100,6002,200,6003,300
UNIT br40a CHAN 2
br40.0
This list file will do the following to input file br40.0:
- insert the event code 6001 into trials 1-100
- insert the event code 6002 into trials 101-200
- insert the event code 6003 into trials 201-300
- no changes to any subsequent trials.
Note that the values 100, 200, and 300 are input trial numbers, not the number of codes to add.
The input trial number is incremented each time a trial is added from anewinput file. Look closely at this list file and its comments:
PARAM TAG 6001 25 6002 35 6003 45 spaces can be used instead of commas
UNIT br35a CHANNEL 2
br35.0 new input file with 25 trials (trial numbers 1-25), trials get tag 6001
br35.1 new file with 10 trials (trial numbers 26-35), trials get tag 6002
UNIT br35b
br35.0 old input file, trial numbering is ignored here, this file keeps tag 6001 on all its trials
br35.2 new file with 10 trials (trial numbers 36-45), trials get 6003
Since br35.0 is repeated, there can be no TAG operation or input trial number change the second time it is used.
Any tag event codes added to br35.0 the first time will remain in the file when it is repeated.
Terminate the tag code operation by leaving the code value empty:
PARAM TAG
The input trial numbers is reset to the value 1 in two different ways.
First, trial numbers are always reset to trial 1 with a new OUTPUT command.
PARAM TAG 6001 25 6002 5000 (The value 5000 is just a big number chosen to be much larger than the number of trials in the files.)
UNIT br35a CHANNEL 2
br35.0 first 25 trials get tagged with event code 6001, remaining trials get event code 6002
br35.1 assuming there are fewer than 5000 trials, all get tagged with event code 6002
OUTPUT br36 -- New output file --
UNIT br36a
br36.0 just as before, first 25 trials get 6001, all the rest get 6002
br36.1
The other way to reset the input trial number is with this command:
PARAM TAG RESET
The TAG RESET command can make tagging much easier.
Without it, you might need to know the exact number of trials in each file.
With the TAG RESET command, you can just reset the count at the start of a new input file:
UNIT br35a CHAN 2
PARAM TAG 6001
br35.0 All trials from this file will start with event code 6001
PARAM TAG 6002
br35.1 All trials from this file with start with event code 6002
PARAM TAG RESET
PARAM TAG 6003,25,6004,5000
br35.2 The first 25 trials will start with 6003, the remaining trials will start with 6004
PARAM TAG RESET
PARAM TAG 6005
br35.3 All of these trials will start with event code 6005
PARAM TAG Turn tagging off
br35.4 None of these trials get tagged.
A tag event code is added at the start of the trial and can be manipulated with the
PARAM IGNORE and PARAM INSERT commands. Using PARAM INSERT, the tag event code
can be moved (actually, duplicated) to a spot later in the trial. See the PARAM INSERT description below.
Trim codes
Every occurrence of this code is removed from the trial except the first occurrence.
PARAM TRIM 116
This example allows only one touch event in each trial.
Original data:
7 3 23 19 7 8 116 116 116 116 116 116 116 116 43 25 7 19 116 116 116 116 55 ......
Trimmed data:
7 3 23 19 7 8 116 43 25 7 19 55 ......
Thin codes
Reduce the frequency of this code by keeping only every 2nd or 3rd or Nth occurrence of the event.
General format:
PARAM THIN <event code> <thinning value>
Example:
PARAM THIN 104 3
This example keeps every 3rd occurrence of event 104. Thus, 2/3rds of the events are discarded.
Original data (event 104 occurs 13 times)
7 3 23 104 104 104 19 7 8 104 104 104 104 104 104 104 116 43 25 7 19 104 104 104 104 55 ......
Thinned data (event 104 occurs 5 times)
7 3 23 104 19 7 8 104 104 104 116 43 25 7 19 104 55 ......
Any number of event codes may be thinned at one time. Include one line in the list file for each event code. Thinning may be stopped for an event by choosing a thinning value of 1 (i.e., keep every event). Thinning is intended to reduce spike frequencies for easier processing and display. Therefore, thinning occurs before spike channel remapping.
Every occurrence of these codes are unconditionally removed from the trial.
PARAM IGNORE 40-42,29,202-340
This example shows a group of event codes that will not be processed..
Note thatMatOFFhas other powerful solutions to deal with circumstances like missing events. Look atHistory scriptingto see how to compensate for missing event codes.
The insert parameter is a means of duplicating a code within a trial. The reason for duplicating codes is to simplify search sequences. If you are searching for a trial that begins with a key event code, and the rest of the key events occur much later in the trial, then you must either do a global ignore of all the intervening codes, or do a complex sequence that includes all possible intervening codes. It is easier to duplicate the early event code by placing it near the other events in the trial.
Here is an example. Event code 51 occurs early in a trial and the rest of the sequence is 14 24 39 184. In-between code 51 and 14 could be many different combinations of codes. To ease searching, we can insert code 51 before code 14. Code 51 is called the "primary code" and code 14 is the "trigger code". To perform this task, we add a PARAM INSERT command in the list file and create an insert file with the following entry:
51 14
Note that the code 51 is inserted immediately before code 14 only on trial were the code 51 occurs before 14. The new code 51 gets the same time stamp as code 14.
More complex insert files can be created. We might like to insert any one of a group of primary codes into a trial, depending which code(s) occurred. In our example, let's suppose codes 50, 51, 52, 53, and 54 specify the type of trial underway, and we would like to place one of those codes (which ever one begins the trial) immediately before code 24 or 25, depending which response the experimental subject makes. An appropriate insert file would look like this:
50 24 25
51 24 25
52 24 25
53 24 25
54 24 25
If this file is named INSERT.C50, then the list file should include:
PARAM INSERT INSERT.C50
When the following trial is encountered, its code sequence is changed in the output file.
If the trial originally has these codes
<50><14><25><28>
| |
| |
| ----- trigger code
----- primary code
The resultingMatOFFfile will get a copy of code 50 immediately before code 25.
<50><14><50><25><28>
The resulting MatOFF file will get a copy of code 50 immediately before code 25.
<50><14><50><25><28>
|
|
------- new entry inserted here
The insert operation does not work on automatically inserted condition codes or the start-of-trial code (198).
Remap files (assigning spike channel numbers)
MatOFF supports only two spikes if the default settings are used. It supports eight spike (pulse) channels with the -8 command line option. MatOFF can support up to 100 channels if a Remap file is used. The Remap file specifies which Cortex events will be used as spikes. MatOFF pulse channels are numbered from 0 to 99.
The default mapping of Cortex event codes onto MatOFF spike channel numbers is:
Cortex event code 1 ===> MatOFF channel 0
Cortex event code 2
===> MatOFF channel 1
This mapping arrangement is equivalent to using a Remap file that has two
entries. The first entry is the event code assigned to channel 0 and the second entry is the event code assigned to channel 1:
1
2
If you want MatOFF channel numbers to match CORTEX channel numbers,
try:
0
1
2
If the -8 command line option is used, the default mapping arrangement
is:
Cortex event code 1 ===> MatOFF channel 0
Cortex event code 2 ===> MatOFF channel 1
Cortex event code 3 ===> MatOFF channel 2
Cortex event code 4 ===> MatOFF channel 3
Cortex event code 5 ===> MatOFF channel 4
Cortex event code 6 ===> MatOFF channel 5
Cortex event code 7 ===> MatOFF channel 6
Cortex event code 8 ===> MatOFF channel 7
An equivalent Remap file would look like this.
1
2
3
4
5
6
7
If you want the Cortex channel numbers to match the MatOFF channel numbers, you can have MatOFF skip channel 0 by placing a dummy value in the first line of the Remap file:
0
1
2
3
4
5
6
7
A Remap file is required if you want to use more than eight spike channels,
or to change the default mapping of spike events. A Remap file is first
created with a text editor, then the file name must given to MatOFF. The file name can be given to MatOFF by specifying the name
with the Setenv command:
set REMAPFILE=<file name>
Alternatively, the REMAP file name can be specified as a PARAM in the LIST
file.
PARAM REMAP <file
name> or
PARAM REMAP
OFF
(same as specifying the default mapping).
Recommended file names are of the form: <base name>.RMP.
The REMAP file format has one integer value per line. The integer is a Cortex
event code. Line 1 maps to MatOFF channel 0, Line 2 maps to MatOFF
channel 1, etc.
0
<=== Dummy entry to skip MatOFF chan
0
1
<=== MatOFF chan
1
3
<=== MatOFF chan
2
5
7
8
9
21
24
<=== MatOFF chan 8
Example of advanced list file
Here is an example of an advanced list file.
PARAM INSERT br35.ins <-- Insert file
PARAM IGNORE 20000-25000 <-- Event codes in this range will be ignored
UNIT br35 CHANNEL 0-3,7 <-- Save spike data for channels 0,1,2,3, and 7. Default to 0
OUTPUT br35p <-- Set the output file name for this and subsequent units
PARAM EOG 200 <-- A/D sample rate of 200 Hz for EOG data
PARAM XGAIN 2.0 <-- Double the gain of the X EOG data
PARAM YGAIN 2.0 <-- Double the gain of the Y EOG data
br35.0
br35.1
PARAM XEOG <-- Exclude EOG data
br35.2
PARAM XEOG OFF <-- Normal EOG processing (excluded for BR35C only)
br35.3
PARAM INSERT OFF <-- Stop inserting codes
PARAM EOG 100 <-- Change A/D rate
PARAM XGAIN 1.0 <-- Return the EOG gains back to normal
PARAM YGAIN 1.0
UNIT br112 CHANNEL 1
OUTPUT br112 <-- Close previous file and start a new output file
br104.1
br104.12
Cortex features supported by the Makdat command
There are several limitations and special considerations that the user needs
to understand.
Converted files have special codes inserted into the data stream to handle
some of the differences between file types:
| Code | Meaning |
| 0 | This code is never used |
| 1-8 | Spike channels 1 to 8 when the "�8" command line option is used. Otherwise, only codes 1 and 2 are reserved for spikes. A REMAP file can be used for processing up to 100 spike channels. |
| 100 | Must mark the start of A/D data for that trial. |
| 101 | Must must mark the end A/D data for that trial. |
| 102 | (Reserved) |
| 103 | (Reserved) |
| 116 | Cortex uses this code to indicate when the touch-screen has been touched. |
| 198 | Automatically inserted at start of new trial. |
| 2xx |
Cortex condition code + 200. The Cortex condition is read from the header of each trial. A value of 200 is added to the condition code, and then inserted into the data file. The user is responsible for not using these codes for any other purpose. If the "-T" command line option is in effect, Cortex does not store condition codes. Codes 200 and above become available as regular event codes. |
| 199 | Automatically inserted at the end of each trial. |
The following limitations are placed on Cortex data files
1) It is up the Cortex State System programmer to place codes into the data
stream to indicated success and failure trials. This information is
not<</em> extracted from the Cortex trial headers.
2) To store A/D data, code 100 must be used to indicate A/D start, and code
101 must be used to indicate A/D stop.
Other limitations
1) Unit names are restricted to 12 characters.
2) Only 32,000 event code values are supported.
3) Only the first 1000 events can be named in an Event Code File.
Early versions of 32-bit Cortex
Early versions of 32 bit Cortex generated data files using a slightly
different header structure than all the other versions of Cortex. At the moment these files are not supported.
Analog channel mapping
Makdat
handles up to four channels of A/D data. Cortex (above V5.7) collects standard X and Y EOG data from channels 3 and 4 on a Metrabyte or Compuboard interface card. Cortex can also collect EPP data on A/D channels 1 and 2.
Makdat and MatOFF preserve these channel numbers.
Graphical User Interface for Makdat
There is a pushbutton on the MatOFF File Menu that brings up a graphical user interface (GUI) for the Makdat command. The user can fill in the blanks on this interface and make checkbox selections. When ready, the user hits the RUN button, which issues a Makdat command based on the selected values.