ML & Bayesian

ML estimator : Choose the most probable hypothesis about parameter space, which is one of the hypothesis.

Bayesian estimator : Hypothesis are used by its prior and its likelihood. The Bayesian estimation of theta are using weighted all the hypothesis(posterior), not the only one with most posterior.

Note: First time to use Formula, Input Formula in visio, export the htm, find the tif file in the htm folder. a little tricky.

belief in ANN

I guess Geoff Hinton's work is just what i want.

Those days i am totally dispointed with the computer or program, how many fantacy things they can do, they just do it in a dummy way, when you want to analyze a simple real question like ECG waveform, it just can't be flexible to handle the different problem. The algorithms like HMM and DTW, SVM, all of those, may be complicated, may be hard to understand, but is just a dead program can't change so much. They just try to give a fixed shoes to the problem.

When doing pattern recogintion, it is the human who choose the features, who figure out what is the most discriminative feature that is useful. In a funny way, this may be called a HCI in a desperate way. You know however complex the algorithm is, It just give up the most hard work to you. And if critisize it for its starborn, it just give you a cold face.

Expect the true AI 's flexibility, I am thinking whether we can get the features by the program. After all, it is just a guessing of function of data.

Bang, I saw this video by Geoff Hinton, such a good idea. By the way, i have the same feeling with the joke he played on SVM. Yes, it is just a perceptron, nothing more.

I have deeply belief in ANN, which has a solid ground of simulating the biology. If there is no God, there is hope, i mean in ANN. And if there is God, there is definitely hope, or hopeless for all sciences.

’The Next Generation of Neural Networks‘

http://www.youtube.com/watch?v=AyzOUbkUf3M&feature=related

http://www.cs.toronto.edu/~hinton/

ECG Processing

Preprocessing

1) DWT

2) High, Low, Bandpass, Moving Average Filtering

3) EMD

4) Adaptive Filtering

QRS detection ( 30years history)

1) Wavelet based Auto-Threshold Method

2) Differential Operation Method

3) MART method

4) Template Matching

5) Automata Method

6) Cross-zeros Method

ECG Segmetation

1) HMM

2) Fiducial Point Finding

ECG classification

1) Wavelet as Features

2) DCT as features

3) ECG waveform PQRST as features

4) DTW

5) ANN

6) Fuzzy & NeuroFuzzy

7) Clustering

Since the ECG is non-stationary waveform, maybe we can view it as a subsection function.

Then the matching or evaluating method will be proposed.

The advantages of the method should be:

1) easy to get the fiducial point,

2) so the analysis can be fulfilled in all of the aspects.

All about Variance and Covariance

To quantify the degree to which the two variables, X and Y, vary together (covary).

It is often important when analyzing x+y or x-y.It is often shows different information compared with its correspondence, variance. It shows whether two variables are related, besides whether the variables change strongly or not( random or not).When covariance is not zero, two variables are related.

reference:

http://www.visualstatistics.net/Visual%20Statistics%20Multimedia/covariance.htm

EM Method

In unsupervised learning, what is lost is the label of each data compared with supervised learning. Which is considered as the missing parameters. This is equal to the problem which EM algorithm trying to solve.

So we first estimate the probability of Labels -> E-Step

Then using this Labels, to compute the ML of parameters -> M-Step

http://www.academicearth.org/courses/machine-learning

Oh!, We got a unsupervised learning for HMM

I am trying to use HMM in ECG processing, which is very popular right now, so i get the " A Tutorial on Hidden Markov Modes and Selected Applications in Speech Recognition" .

Then i got some open source on the websites.

Then what i need is to train it, even i have read the ML training, i still think i should give training data with observations and its responsible states, i.e. (O, S) pair. That is why i try to use matlab UI to build a software to let me annotate the ECG waveform to give the (O, S) pair.

After I ve done that, i find out" There are two training method", the supervised one and unsupervised one. The first one just use the frequency to replace the probabilities. Well the second one just try to improve the parameters to make observations more likely to happen.

So it is not smart to do things too fast, right?

Matlab UI

Root is screen, Figure is part of screen, Axes is part of Figure, where you can draw image, line, rectangle, surface, and text objects.

1) Axes is plural of axis

2) Axes, Menu, Uiobjects are children of Figure;

3) Root can not be create and destroyed.

some functions used first time

1) handles -> used for transfer data between sub-functions.

2) set(hObject, 'toolbar','figure') -> add a toolbar for figure control such as zoom

3) guidata(hObject, handles) -> save changes to guidata stored in handles, if u do not do this after you modifying the handles, it will be reset after you exit the sub-function

4) dcmobject = datacursormode(gcf) -> get info of data cursor setups, you can use

5) getCursorInfo(dcm_obj) to get position of current cursor, useful for me!

6) uigetfile -> open a dialog to let you select a file, and return its file name and path

7) uigetdir -> open a dialog to let you choose a directory.

8) ginput -> let you choose a point in the figure using cross mouse pointer.

9) gtext -> let you put a text on the figure using cross mouse pointer.

add toolbox

It is embarrassing to never use a external toolbox,

addpath(genpath('c:/data'));--ADDPATH DIRNAME prepends the specified directory to the current

savepath(genpath('c:/data'));-------SAVEPATH saves the current MATLABPATH in the pathdef.m, which was read on startup.

genpath('c:/data') return a path string, plus, recursively, all the subdirectories of D, including empty subdirectories.

open c:\data.m

text() in Matlab

when i use this code in Matlab,

text( x, y, 'N', 'Color', 'r');

i get an annoying error

??? Error using ==> text

Invalid parameter/value pair arguments.

finally i found answer on the web,

it is because x is a integer, it must be double.

then i convert x into double class, then it is ok,

damn machines and thank google

template matching

It is maybe the simplest method for classification, but still with some tricky problem. We can make it powerful and robust by making the template a model instead of a samples set.

1) in DTW, the sample time is a parameter of the model, which can be changed. It has brought great improvement to the effect, and was taken by the Speech Recognition group at the first few years.

2) for spike interference, we can a) use some orthogonality. b) split the template into pieces, d) check the sigma = sum(T-x)^2 . e) make the template a model with its parameter changeable and select the parameter to minimized the sigma, which is the most powerful method.

3) In some way, we can view the HMM as a template model, which can generate a lot of templates of different class.

note : thanks for Akash Kansagra

one of my idea: symbol DTW，feature match as amplitude to use DTW

Bring context to bear

Excuse me, another way to say [utilize] is [take ... into bearing]? help me. [No, It is "Bring ... to bear"]. OK, let us go back to work.

When we analyze uncertain things, we usually do not use the routine of getting the features first and then analyzing them, because that is hard, that is kind of clustering, when the dimention is more than 2, no human can do a good job. So we always use the routine of getting some features first, coming out with some proposition, and looking for evidence to prove our proposition is right or wrong. And in that routine features used to get the proposition is a start point with some probability. The evidence is Context into bearing.

Well it is intuitive, but it is hard to use in AI, Actually we are not thinking so accurately, so we find it is hard to define the amount of information of evidence, the relationship of evidence and the indepence of all the evidence. And the start point come to our head randomly, which make it a little hard to be copied.

HMM, Bayesian, Fuzzy logic, Evidence Theory, Possibility Theory and all the Fusion Theory and Filter Theory are used to handle this problem, HMM is a simplification of Bayesian. And Bayesian seem to lie in the theory and mathematical realm. Filter Theory ignores the give the relationship a fixed outline in time view, gives a obscure model. Fuzzy logic, Evidence Theory and Possibility Theory are all trying to decline the responsibility by just simulating the human. Fusion is just another way to look at all this things.

In the hello-world example, we are trying to say how possible the rain will stop Peter to go shopping, some say 0.8. some say 0.5. All this stuff make yourself not so confident and seriously doubt the correctness of your decision.

I guess that is why we are trying to train our guessing machines. and that is usually a task of ANN, which in its birth getting the knowledge from data, which is similar to humans. Finnaly this leads to the so-called research of Maching Learning, which is trying to extend the training idea, and trying to not so human-like.

Polygonal Approximation of Curves

Definition: Using line segments to approximate one curve. No doubt it will compress data and simplify the shape analysis.

Methods

1) Heuristic

Search point by point, until the error boundary has been exceeded. It just gives out the line segments which does not exceed the Error Threshold.

2) Optimized

Use a shortest path theory to solve the problem. The path length will be the errors, and the path is the selected vertex, then use a optimized method to search the best path. It gives out the line segments Minimizing the total error.

Problem I met when i used it:

1) Noise

when there is noise in signal , most of the time, the algorithm can get the mean line of the noise, which can be seen as a denoising effect. But some time it just takes some big noise peaks as the true line of the signal.

some conditions to identify this: a) noise in context as reference, b) at least including one up and down shapes, c) errors is zeros, d) length is short but not so short, e) slope is high.

2) computational burden

the heuristic method is fast, but can give a -peasudo-best results, and can not give the corners good thinking.

the optimized method need too much searching loops.

Empirical Mode Decomposition-EMD

EMD was proposed by Norden E. Huang, and used for non-stationary and non-linear signal analysis.

Intuitively if we connect all the maxima of one signal, we will get its envelope, the same is true for minima. then signal can be decomposed into the mean of the two envelopes and the residue. If we do this operation continually, then we will finally get a signal without riding waves(baseline is zero, i.e. mean of envelopes will be zero).

1. EMD is a data driven algorithm, it decomposes the signal into IMF, which is not a pre-defined signal. IMF is generated by data itself.

2. The idea is : the signal is decomposed into local oscillations and its residues(can be viewed as a baseline) in some scale(it is not fixed). The osillations is high frequency part resides on the low frequency residues. If we do the same decomposition on the residues, then we can get another osillations with wider scale.

3. The algorithm is based on finding oscillations using local extrema.

1) identify all extrema of x(t)

2) interpolate between minima(resp. maxima), ending up with some envelope e_min(resp. e_max)

3) compute the mean m(t) = (e_min+e_max)/2

4) extract the detail d(t) = x(t)-m(t)

5)Iterate steps 1)-4) on the residual until the detail signal d(t) can be considered an IMF: c(t)=d(t).

6) iterate 1)-5) on the residual m(t) to extract all the IMFs. The stop criteria will be: the signal is constant or monotonic, or with only one extrema.

Definition of IMF: Functions with equal number of extrema and zero-crossings, and must be zero-mean defined by its maximas and minimas. This is for consideration of removing riding wave or baseline, and it has a good Hilbert Transform results, that is it is an good AM-FM signal in the form of a(t)exp( i*theta(t) ), and for IMF, its envelope defined by HT will make sense.

Intermittent Test: To select time scale(shorter than this will not be included) to avoid oscillations with different time scale(frequency) mixed with each other into same IMF.

How to use EMD:

1. EMD of signal

2. Spectral Analysis of IMFs

3. Selection of IMFs based on Spectral characteristic.

4. Reconstruction or partial reconstruction of signal for analysis.

hello, world

it is random for me to check this, but i hope its impact on me, some people and future is non-trivial.

Since i am working on some pattern recognition and expert system, I guess this will be a good start and can record this procedure and maybe some day it will become my homepage.