Forum

EMebane · Joined: Apr 15, 2019 Posts: 124

This resurfaces in my mind every so often so I thought I'd submit it to the feature request pool for consideration, especially if adding a few new algorithms is easy. _Low priority_

I'm sometimes interested in excluding outliers from a collection of numbers. RMS, Variance all exaggerate edge values. Average includes outliers. Etc.

There might be value in adding calculations like the following to TimeTransform to avoid biasing the edge values in a collection of values.

- Median Absolute Deviation (MAD)- standard deviation-like, with influence of values treated linearly instead of squared
- Mean Absolute Error (MAE) - variance similarly made linear
- Interquartile Range

A checkbox to apply edge value removal from the data before proceeding with any of the TimeTransform operations:
- 1.5xIQR Rule (Q1-1.5xIQR to find lower fence, excluding values below it. remove upper outliers similarly with Q3+1.5xIQR)

Moreover, working with just the middle bins of quartile/quintile calculations also comes to mind sometimes, though I don't have a specific use case in mind for these segmented values/bins.

jarek · Joined: Oct 22, 2007 Posts: 1087

Thanks.

We will add Median Absolute Deviation. And Interquartile Range, although this was already possible with Percentile2.

The TimeTransform doesn't remove data. One possibility might be to use ExpressionEvaluator (with SkipOutput variable) which takes input from TimeTransform like a threshold.

EMebane · Joined: Apr 15, 2019 Posts: 124

When one uses SkipOutput, how are the calculations in the element used? I'm not sure what to do with an ExpressionEvaluator that doesn't output anything. Does it have powers other than outputting values to its Out ports?

Thanks for adding the new calculations. Percentile2 is new-ish and I didn't realize it was available.

jarek · Joined: Oct 22, 2007 Posts: 1087

Your expression doesn't send out anything, it only sets the values.

If you don't set SkipOutput, then the output will be sent out. Possibly previous value for every input sample (the expression calculation is triggered for any input sample) if you don't do anything (e.g. "return" in the first line).

The expression execution will happen as usual even with SkipOutput, for example you can assign something to member variables (defined in Declaration section) for later use.