Pipes are the end points used to connect elements. There are many different pipe types (scalar, vector, object etc). Scalar stream value (up to 32 bit) can be considered as a vector with just one element. Object stream contains objects (e.g. texts) which are exchanged by reference (not by value like in other streams).

· Logical pipe is based on scalar pipe (they can be connected each to other). The 0 value is FALSE, and 1 is TRUE. This pipe operates on logical state rather then logical stream. This means, that logical elements usually send only the final logical value based on all available input.

· Complex pipe is compatible with vector pipe. Two complex vectors are stored in one: lower half part contains real fields and higher half part contains imaginary values. It is possible to use vector elements to process complex vectors.

· Matrix pipe is compatible with vector stream. Each matrix is stored as a vector, and matrix parameters (width, height) are stored in MatrixSignalParameters.

There are input pipes and output pipes. One output pipe can be connected to many input pipes. Input pipe is usually connected to only one output pipe. If input is connected to many outputs then it works as multiplexer, but without known order. Each input pipe has internal buffer.

Vector streams

Aside from streamed data, BioEra can also process vectors (one or two dimensional arrays) in real time. This is very unique feature not found in other software.

There are many uses of vector streams. One example is filtering. In conventional applications, input stream is being filtered with various types of digital filters. For example to get amplitude of a brainwave a filter is applied on the signal and the amplitude of the filtered output is measured. In BioEra signal can be first converted to vector (for example with FFT or Wavelet transform or any other), and then all further processing can be done on vectors. This allows build advanced protocols on FFT or wavelets.

One-dimensional vector streams have good support in BioEra. Matrix streams (two-dimensional) were added later have less support.

Event inputs/outputs

Each element in BioEra has special input/output (located at the bottom of each element) for special purpose. Usually they are connected to blue outputs/inputs of an element which allows such special functions e.g. PropertySetter.

Filters

BioEra supports traditional digital filters. There is one implementation which uses Fidlib.

Compare filters with FFT

Some comparisons between FFT filtering (on vectors) and traditional filters were done, and either method has its unique advantages. Traditional filters provide more options and don’t require windowing; the FFT filtering produces more information (entire frequency spectrum) at the time. The overall results can be similar.

An example comparison between traditional Butterworth 4-th order filter and FFT filter (on vectors) is presented here. The input signal comes from quasi random generator. Both filters are set at 7 to 11Hz. Each outcome is averaged over one second. The bottom left chart shows JTF characteristics from FFT filter (green line) and Butterworth filter (green). Although they are not exactly the same, all trends and amplitudes are quite similar.

Controls

BioEra Pro contains controls – advanced elements which let the user to modify current processing settings. They are put on Runtime window into toolbar, menu etc.

· ComboToolbarControl – combo component to select one value out of many.

· IconToolbarControl – icon that can be used as button to trigger action on current design.

· MenuList – automatically creates list of submenus, only one sub-menu can be chosen at a time.

· MenuItemControl, MenuItemList – creates a menu field which can then be used to trigger an action on current design.

Interactors

BioEra Pro contains several interactors – advanced elements for operations and communication with system.

· KeyInteractor – keyboard activity can be generated here

· MouseInteractor - mouse activity (buttons and movement)

· SystemInteractor – to stop, start, pause, save etc.

· SystemEventSource – to receive various system events about start, stop, pause, resume, init, errors and others.

· ElementIntractor – various operations on single element (activate, deactivate, process)

· DesignInteractor – load another design

· CustomInteractor – load/save file path

· OSInfo – get the list of available drives or files in folder

· OSInteractor – execute external application

Signal parameters on element level

Information about signal parameters like rate, range or number of bits is provided on element level (as oppose to global level). An element can retrieve these values from other element connected to one of its inputs. An element can also modify its own values if necessarily. For example if an FFT transform is being done 4 times per second, then it means that the output rate is 4, and this value is returned when rate value is requested.

Element status

Each element can provide status (being updated dynamically during processing). Status is located at the bottom of the element icon (on designer). At the moment status is represented as set of letters with different colors. The red letter X is generic for all elements, it means, that the element has been deactivated and is not processing data. Other letters are defined specifically for each element

and described with element information.

Element array properties

Some element properties contain a set of values. They are enclosed between "[" and "]" characters and are called array properties. Array properties use some characters exclusively, they can't then used directly by the user. Below is the list of characters along with meta values which can be used to encode (and then show) those special codes. Please note: <space> can be used only at the beginning or the end of the text (or alone).

,	<comma>
;	<semicolon>
.	<period>

[

<open>

]	<close>
	<space>

Scales

BioEra is supposed to support all kinds of signals (not only neurofeedback). So there are many different signals and it is not possible to have just one measurement unit. That is why BioEra supports several scales. For example for EEG devices operate on uV (micro-volts), for EMG it is mV (mili-volts) etc. BioEra can operate on amplitude range, frequency domain and time domain.

Amplitude

Signal amplitude can be scaled on many ways. Often data that comes from input device is scaled in micro-volts. There are also other scales: digital and percentage. There are 6 possible amplitude scales available in BioEra at this moment:

· microvolt or uV – microvolt values starting from 0 and more

· digital or digi – digital values starting from 0 and more

· percent or % – percent values from 0 to 100

· balanced microvolt or =uV – balanced microvolt values, any range is possible
(although usually they will be within max range)

· balanced digital or =digi – balanced digital values, any range is possible
(although usually they will be within max range)

· balanced percent or =% – balanced percent values from -100 to 100

Note: there will be other scales added gradually here, e.g. radians, decibels. Mili-volts etc.

Microvolt

At this moment most EEG devices operate on micro-volts (or fraction of microvolt). Other biofeedback devices (like EMG or ECG) may use values of higher amplitudes.

Digital

Digital scale represents direct integer values being received from biofeedback device. BioEra operated on 32 bit numbers.

Percent

Percent scale is relative to full range allowed in an element. Range is from -100% to +100% for numbers symmetrical to zero, or from 0 to 100% for positive numbers.

Decibels

Not implemented yet.

Custom units:
Sometimes input signal can be scaled with unusual units (for example Hertz or degrees). This is not yet fully implemented.

Time

Time scales are at this moment presented usually in seconds. Eventually more units will be provided. Now the values are relative (since the recording start); eventually they will be option to show absolute times.

Absolute time

Not implemented yet.

Relative time in other units

It is implemented for all basic time units (seconds, minutes, milliseconds, hours, days).

Number of samples

This can be used when the rate of the input is constant.

Processing and initialization order

Elements are processed in order; next element is being processed after previous element finished processing.

Elements are being processed in the same order as they were initialized. Elements that are closer to input (signal source) are initialized and processed first.

Access to serial port

BioEra installation package includes all necessary drivers to access serial port on Windows. On Linux there is a driver available http://www.rxtx.org/ that is supported in BioEra (but not included). For further information about serial port installation or problems with serial port access see Sun’s site: http://java.sun.com/products/javacomm/downloads/index.html.

Mac OS X

On Mac OS X the RXTX driver has been confirmed to work properly. Serial driver installation process may depend on USB-to-Serial adaptor used. Here is more information about example installation that worked well.

Command line options

If BioEra is started without a command line options, then it will open with full edit and runtime options (and the last edited design). Command line options can alter the configuration before start. Command line syntax:

bioera.exe <-or|-op> [-diagnostic] [-log <filename>] [-start] [-root <folder_path>] [-logs <folder_path>] [-design <design_file.bpd>]

Where:

· -or: runtime mode only (no designer), runtime window is shown, and processing is started automatically after start.

· -op: processing mode, design is started automatically after start (no graphic windows)

· -diagnostic: set diagnostic mode (printing messages on console).

· -log <filename>: text messages are saved to this file (and console)

· -start: processing is started automatically after start

· -root <folder_path>: path to BioEra folder, should be provided if started from a different folder.

· -logs <folder_path>: path to folder where logs are stored; default is bioerapro/logs.

· -priorityNormal: BioEra starts with Normal program priority.

· -priorityAbove: BioEra starts with Above Normal program priority.

· -priorityHigh: BioEra starts with High priority.

· -dx – force to use DirectX graphic acceleration. By default BioEra uses OpenGL library for 3D graphics. If OpenGl is not available, then it automatically switches to DirectX. But on some older computers OpenGl drivers may be improperly configured; in such case it will not be able to switch to DirectX by itself. This option forces it.

· -mxNNNN – set maximum memory, where NNNN is a number of Mega Bytes. The default value is 768MB.

· -mnNNNN – set minimum (start) memory, where NNNN is a number of Mega Bytes. The default value is 128MB.

Averaging functions

Very often it is necessary to average results over a time period (ScalarTimeTransform, VectorTimeTransform, VectorInstantTransform) or to show on chart as a single line (VectorToScalar). Therefore it is important to understand how different averaging functions work. Here are example charts that show a few averaging functions on the same input signal:

· JTF signal of delta (1-4Hz), FFT on 512 points (2 seconds with 256Hz input)

· JTF signal of delta (1-4Hz), FFT on 1024 points (4 seconds with 256Hz input) – this result comes from the same input signal, but has 2 seconds delay when comparing to the above.

Video

In version 1.0.9 BioEra provided ability to display and record video. Since version 1.1 it is also possible to edit video in real time (but this is quite limited). Although this feature is beyond the scope of traditional biofeedback, it was added here for number of reasons:

· video feedback (or entrainment) became important part of biofeedback,

· video recording maybe very useful with a research during sleep or with closed eyes

Video editing added in version 1.1 makes BioEra simple yet powerful tool not only for biofeedback but also for other type of research when video analyzing is required (e.g. robotics, face recognition, artificial intelligence etc).

Scalar and float values

BioEra uses integer (scaled) numbers in all Scalar elements. Values in scalar pipes are also exchanged as integers. The primary reason is performance, especially on devices like PDA without coprocessor.

Float numbers are used in all Float elements and Float pipes. There is also an optional DoubleFloat type which can be used for ultra high precision at the cost of slower processing (lower performance).

Scaling

Scalar numbers are exchanged as integers and scaled relative to the “Digital Max” and “Physical Max” which can be set individually for each element (can be viewed in Advanced Properties). The float value is calculated as: ScalarNumber * PhysicalMax / DigitalMax. For example if the scalar number is 100, DigiMax=1000 and PhysMax=1, then the actual value is 0.1. The raw scalar number can be viewed on Console with Debugger element.

Because signal properties can be different in each element, then some elements (e.g. Threshold, BarDisplay) have options to “Rescale”. This option should be selected if DigitalMax/PhysicalMax ratio is different in the elements which are connected to inputs.

There is no scaling for Float numbers. If the above seems too complicated, you may want to try to use equivalent Float elements, which should behave the same, but have not been tested as well.

Border

Some graphical components like toolbar have option to set custom border which is loaded from an image. Border can be either horizontal (image width is 1 pixel), or only vertical (image height is 1 pixel) or both. When border is painted the four quarters of border image are used for each border line, and four image corners for border corners. Border image can contain transparent pixels.

Example borders are in folder images/border.

Nested designs

Nested design can be added to existing design and behave like an element. It has inputs and outputs and properties.

Create nested design

Nested design is created and opened the same as normal design. It contains however some special elements which customize its behavior.

The most important are: NestedInputs and NestedOutputs. You must have at least one of them (or both) in each nested design. They are used to define inputs/outputs which are used as interface to this nested design from main design.

Initially each input/output pipe on NestedInputs/NestedOutputs is black, this means their type is not defined so can be connected to any pipe. Once it has been connected, it automatically converts to compatible pipe. Note: only connected inputs/outputs are shown when this nested design is added to the main design.

NestedProperties is used to customize the behavior of the nested design. For more details see the description of this element.

Add nested design

To add a nested design you need first to add NestedDesign element. Then in properties set path to the nested design file.

Nested charts

One of the options in NestedProperties is ‘Charts are nested into one chart’. If this option is selected, then this nested design will expose only one chart to outside world. This chart will contain all internal charts. For more details see the description of NestedProperties element.

Colors

Color properties dialog

Many elements allow set colors. Color is usually selected in the Color Dialog. BioEra supports transparency whenever possible.

· Four sliders show the current color selection. Sliders listed from top to bottom:

o Red – red component

o Green – red component

o Blue – red component

o Alpha – transparency component. The 0 value is full transparency and 255 means the color is opaque. Note: the 0 can mean also default value, see below.

· Fully transparent color – if selected, then the color is fully transparent. Can be used to “disable” some colors, e.g. name color set as transparent makes it invisible.

· Default color – if selected then this color is expected to have a default value. Default value depends on context, there is no general rule, usually this is a system color for this particular component (e.g. menu color the same as system menu color)

· Text field – it shows the color components in hexadecimal form. FF is maximum (equal to 255) value and 00 is minimal value. Note: alpha component is first, then red, green and blue.

Gradient color

Gradient color (for more information about this topic see here) allows convert a value into a color hue. For example in Polygraph it is possible to fill the area under the trace using gradient color. Gradient color is loaded from an image file and can be custom created. Several gradient color images are under images/gradient folder.

Signal properties

Each output pipe of each element can have its own signal properties. The signal properties describe the signal which is being processed by this pipe. Different signal parameters occur when there are different signal sources, for example a few input devices, EEG and GSR at the same time.

To view Signal properties of an element, right click on the element, select Advanced Properties and select tab Signal properties.

· Rate – defines how many samples are exchanged per second.

· Digital min, Digial max – samples are usually exchanged as digital numbers. Those settings show its default range.

· Physical min, Physical max – corresponding physical values for the above digital numbers.

· Encoding – currently not used

· Physical unit – physical unit of the signal e.g. uV (microvolt) or Ohm or s.

· Vector length – used in vector pipes, defines the size of the vector

· Vector max, Vector min – defines the physical value of the vector at each side

· Vector unit – physical unit of the vector, .e.g. this is Hz in the output of FFT.

· Vector fields descriptions – each field in the a vector can contain a description

· Info – additional information

Java platofrm

Almost everything in BioEra is created using Java. Java has been selected as the primary development platform for BioEra because:

· It is proven and reliable technology, used for instance by NASA on Mars.

· It is the easiest and the most popular solution for portability. We can run BioEra on PDA, Mac and Linux.

· Java development is very effective for rapid improvements.

· Although the performance of Java can be sometimes less then C language (see performance chapter below), with JIT (Just in Time) virtual machine, the speed is reasonable in most cases. External C libraries (like numerical recipes) can be reused and included easily with JNI interface.

· If properly done, then the same code does run on many different systems without any modifications. This can depend heavily on JVM (Java Virtual Machine) chosen, it is recommended to use Sun’s JVM whenever possible. Although they are not always the fastest, they provide relatively consistent environment.

· Java is perfect for elegant object oriented development that can be easily extended.

There are some additional components which were created in C++ for best compatibility with Windows OS e.g. Video, DVD and Flash players.

Java performance

Java has a false reputation of being slower then other languages like C or C++. Do not believe it! That was the case with first versions of Java long time ago (10 years ago), but it has been improved dramatically in recent versions because of the JIT compilers (which compile to native code on the fly). Due to the optimizations many functions are now faster then in native C programs, some are slower, overall performance is very comparable. There are many research done and published on Internet, e.g.:

· http://en.wikipedia.org/wiki/Java_performance

On PDA or Smartphone

Initially BioEra was designed to work on PDA (Personal Digital Assistant). PDA is a great opportunity for portable biofeedback, handheld devices are already very powerful (comparable to PC from before a few years) and they will be faster.

Currently only some devices can be used with PDA version:

· PET – bluetooth – available on most PDAs

· Nonin4100 – bluetooth – available on most PDAs

· WiTilt - bluetooth – available on most PDAs

· ModEEG – possible only on some PDA which have serial port connector

Note: PDA version has not been released as commercial product yet.

Connecting BioEra with other platforms and programs

BioEra supports various methods to allow interfacing with other applications and languages.

o CustomElement – allows interfacing with another java program. A custom element can be created that way and then added in Designer

o XmlNetServer – this interface uses plain Xml text and is language independent (can be used with any programming language)

o .NET SDK – an SDK for Microsoft .NET is provided and supported. It can be used from inside a CSharp or Visual Basic .NET application. C# example and source is provided which demonstrate how to receive data from BioEra (connecting to XmlNetServer). See picture. More info about SDK.

o ApplicationPlayer can be used to start and control an executable application.

o MIDI element can be used to play on external MIDI device like electric Piano.

o SerialPort element provides two way communications via serial port.

Pipes