Failure Sensor Detection by Pattern Comparison on Time Series

Purpose

Find the interesting patterns in the time series data in order to detect the failure sensors.

Process

Time series data source. We leveraged Databricks and delta table to store our source data which includes sensor_id, timestamp, feature_id, feature_value. To make it run quicker, you may aggregate the time to hour or day level.
In order to vectorize the features, the features in the rows have to be converted into columns.
Clean outlier. Even the data source is spotless on the aggregation level. There may still have some outliers, like extremely high or low value. We set 5 STDs to identify and clean these outliers by per sensor ID.
Normalization. Use max_min function to normalize selected columns. You can either use max_min normalization for the features based on all sensors or by each sensor, depends on how variety between these sensor data.
Pattern comparison. We use an existing reference pattern to compare the time series from very left to very right time point. For example, list [1,1,1,1,0,0,1,1,1,1] (visualization: ) has 10 points, it slides on 30 points’ time series, you will get 20 comparison results. There are two ways to compare two time series.
- Pearson correlation.
- Euclidean distance.
Filters. The result of comparison will give you how similarity of all-time points. You can set the threshold for this similarity value to filter non-interesting points. Also, you can set how frequency of pattern happened as threshold to further filter. Here is an example of threshold.
- corr_ep_thredhold= 0.8 (similarity greater than 0.8)
- freq_days_thredhold = 20 (the pattern happened more than 20 times in all time series)
- freq_thredhold = 40% (from this pattern starts to ends, more than 40% days it happened)
The result you can persist into a database or table.

Pros

Since the feature is normalized, the comparison is based on the sharp rather than absolute value.
Statistic comparison is quick, special running on the spark.
The result is easy to understand and apply to configured filters.

Cons

Right now, the Pearson correlation and Euclidean distance only apply to one feature in a time. We need to figure out a way to calculate the similarity on the higher dimensions in order to run all features in a matrix.
Pattern discovery. To find the existing pattern, we based on experience. However, there is an automatic way to discover pattern. Stumpy is a python library to do so. It can also run on GPU.

Future work

How to leverage steaming to find the pattern immediately once it happens, rather than the batch process.
High dimension matrixes comparison.
Associate Stumpy to find interesting pattern automatically, then apply the new patterns into configured filters.

Eiganvectors from Eiganvalues

This is a unbelievable discovery from PETER B. DENTON, STEPHEN J. PARKE, TERENCE TAO, AND XINING ZHANG. The original paper you can find here. In a short nut, we can get eiganvector through eiganvalues only. This magic formular is like blew:

We only need to know eiganvalues in original matrix then we can calculate its eiganvectors through sub-matrix.

Currently, I don’t know or image what will effect on the road, but the eiganX is the basic for AI, dimension reduction, feature extraction, etc. It also may help us improve the speed to get eiganvector if we need to incrementally add data on a known matrix.

I wrote a very simple python script to prove this formula(surely, it is correct). I think it can archive by GPU as well.

import numpy as np
from numpy import linalg as LA

matrix_size = 6
org_x = np.diag(range(1,matrix_size+1))
w, v = LA.eig(org_x)


print("orgnal matriax: \n %s \n" % org_x)
print("eigan values: \n %s \n" % w)
print("normalized eigenvectors: \n %s \n" % v)

print("START NEW ALGORITHM \n")

result=[]
for _ in range(matrix_size):
    result.append(0)

for n in range(matrix_size):
    for j in range(matrix_size):
        sub_x = np.delete(org_x,j,axis=0)
        sub_x = np.delete(sub_x,j,axis=1)
        w1,v1 = LA.eig(sub_x)

        # in term of new formula to get orignal matrix eigenvecotr through eiganvalue
        numberator = 1
        denominator = 1
        for i in range(matrix_size-1):
            temp_n = w[n] - w1[i]
            numberator = numberator*temp_n

        for i in range(matrix_size):
            if(i!=n):
                temp_d = w[n] - w[i]
                denominator = denominator*temp_d
        result[j] = numberator/denominator

    print("%s \n" % result)

result:

orgnal matriax:
 [[1 0 0 0 0 0]
 [0 2 0 0 0 0]
 [0 0 3 0 0 0]
 [0 0 0 4 0 0]
 [0 0 0 0 5 0]
 [0 0 0 0 0 6]]

eigan values:
 [1. 2. 3. 4. 5. 6.]

normalized eigenvectors:
 [[1. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 0. 1.]]

START NEW ALGORITHM

[1.0, -0.0, -0.0, -0.0, -0.0, -0.0]

[0.0, 1.0, -0.0, -0.0, -0.0, -0.0]

[0.0, 0.0, 1.0, -0.0, -0.0, -0.0]

[0.0, 0.0, 0.0, 1.0, -0.0, -0.0]

[0.0, 0.0, 0.0, 0.0, 1.0, -0.0]

[0.0, 0.0, 0.0, 0.0, 0.0, 1.0]

Reference:

EIGENVECTORS FROM EIGENVALUES, https://arxiv.org/pdf/1908.03795.pdf

Tao’s Blog, https://terrytao.wordpress.com/2019/08/13/eigenvectors-from-eigenvalues/

Raspberry PI Security

Lots of ready-to-use opensource project can be found on internet for raspberry PI object detection. Most of them can do very well to motion detection or object classification. I am thinking how to merge them together to make a practical security system that can help all of us to make our home safety. Since It is open source, I will share my design, source code and project milestone. ( mostly to push myself to finish it eventually 🙂

Design:

start camera to capture each frame
save the first frame as reference. and this reference frame will be replaced every 5 mins when there is no movement detected.
continuously compare the current frame with the reference frame.
if any movement detected, draw some interesting areas.
loop these interesting areas, if its size out of threshold, move it to next DNN network.
use DNN network to classify the object.
if object is human, trigger dedicated process “Event process”. includes, recording video, send notification and make speaker noise.
loop to the next frame.

Milestone:

Oct 10th-15th: finish basic function to detect movement, recording video and send mail.

Oct 16th-22th: add DNN network

Oct 22th-30th: add speaker and integration test.

Potential update:

GPU accelerate on Cuda device
GUI
restful API

Source:

https://github.com/neoaksa/raspiberry-security

Demo(till Oct 15th):

Reference:

Deep learning: How OpenCV’s blobFromImage works. https://www.pyimagesearch.com/2017/11/06/deep-learning-opencvs-blobfromimage-works/

Raspberry Pi: Deep learning object detection with OpenCV. https://www.pyimagesearch.com/2017/10/16/raspberry-pi-deep-learning-object-detection-with-opencv/

how to install opencv on the raspberry pi 3 Model b+ (with camera) https://pysource.com/2018/10/31/raspberry-pi-3-and-opencv-3-installation-tutorial/

Home surveillance and motion detection with the Raspberry Pi, Python, OpenCV, and Dropbox. https://www.pyimagesearch.com/2015/06/01/home-surveillance-and-motion-detection-with-the-raspberry-pi-python-and-opencv/

Night Sight with Google Camera

As an amateur photographer, I was believing DSL is better than phone camera since it has a much larger CMOS so that it can receive more photons, until I installed Google Camera on my Galaxy S8. The results surprised me a lot. I mean, it performed better than my Conan 5DIII in the most of case without editing in the software like photoshop.

Except HDR+ and portrait mode, Google camera provides a magic mode called Night Sight. Unfortunately, this can only work on Pixel series (Pixel 3/3a is the best) phone with hardware support. You can find the A/B Testing here.

left: disable Night Sight; right: enable the Night Sight.

How does Night Sight improve the quality of shots in the night?

HDR+. HDR+ is the foundational function for Night Sight. It is a computational photography technology that improves this situation by capturing a burst of frames, aligning the frames in software, and merging them together. Since each frame is short enough to prevent the blur caused by hand shake, the result turns out to be sharper and wilder dynamic range than without HDR+.

This image has an empty alt attribute; its file name is IMG_20190518_101130.jpg — **with HDR+, we can clear see the background in an indoor place**

Positive-shutter-lag (PSL) . In the regular mode, Google camera uses zero-shutter-lag (ZSL) protocol which limits exposures to at most 66ms no matter how dim the scene is, and allows our viewfinder to keep up a display rate of at least 15 frames per second. Using PSL means you need to hold still for a short time after pressing the shutter, but it allows the use of longer exposures, thereby improving SNR at much lower brightness levels.
Motion metering. Optical image stabilization (OIS) is widely used in many devices to prevent hand shake. But it doesn’t help for long exposure and motion object. Pixel 3 adds motion metering to detect the motion object and adjust the exposure time for each frame. For example, if it detects a dog moving in the frame or we are using the tripod, it will increase exposure time.

Super Res Zoom. HDR essentially uses algorithm to aliment and merge the frames to increase the SNR( signal to noise ratio). Pixel 3 provides a new algorithm called Super Res Zoom for super-resolution and reduce noise, since it averages multiple images together. Super Res Zoom produces better results for some nighttime scenes than HDR+, but it requires the faster processor of the Pixel 3.

Learning-based AWB algorithm. When it is dim, AWB( auto white balance) is not functional well. And it is an ill-posed problem, which means we cannot inverse the problem (find out the real color of object in the dark). In this case, Google camera utilizes machine learning to “guess” what is the true color and shift the white balance.

S-curve into our tone mapping. As we know, if we exposure a picture for a long time, all the objects become brighter so that we can not figure out when this picture takes. Google uses sigmoid function to adjust the object brightness ( dark objects become darker, light objects become brighter).

Reference: Night Sight: Seeing in the Dark on Pixel Phones, https://ai.googleblog.com/2018/11/night-sight-seeing-in-dark-on-pixel.html

Math in Machine Learning

Linear Algebra

mathematics of data: multivariate, least square, variance, covariance, PCA
equotion: y = $A \cdot b$ , where A is a matrix, b is a vector of depency variable
application in ML
1. Dataset and Data Files
2. Images and Photographs
3. One Hot Encoding: A one hot encoding is a representation of categorical variables as binary vectors. encoded = to_categorical(data)
4. Linear Regression. L1 and L2
5. Regularization
6. Principal Component Analysis. PCA
7. Singular-Value Decomposition. SVD. M=U*S*V
8. Latent Semantic Analysis. LSA typically, we use tf-idf rather than number of terms. Through SVD, we know the different docments with same topic or the different terms with same topic
9. Recommender Systems.
10. Deep Learning

Numpy

array broadcasting
1. add a scalar or one dimension matrix to another matrix. $y = A + b$ where b is broadcated.
2. it oly works when when the shape of each dimension in the arrays are equal or one has the dimension size of 1.
3. The dimensions are considered in reverse order, starting with the trailing dimension;

Matrice

Vector
1. lower letter. $\upsilon = (\upsilon1, \upsilon2, \upsilon_3)$
2. Addtion, Substruction
3. Multiplication, Divsion(Same length) a*b or $a / b$
4. Dot product: $a\cdot b$
Vector Norm
1. Defination: the length of vector
2. L1. Manhattan Norm. $L1(\upsilon)=|a1| + |a2| + |a3|$ python: norm(vector, 1) . Keep coeffiencents of model samll
3. L2. Euclidean Norm. $L2(\upsilon)=\sqrt(a1^2+a2^2+a3^2)$ python: norm(vector)
4. Max Norm. $Lmax=max(a1,a2,a3)$ python: norm(vector, inf)
Matrices
1. upper letter. $A=((a{1,1},a{1,2}),(a{2,1},a{2,2}) )$
2. Addtion, substruction(same dimension)
3. Multiplication, Divsion( same dimension)
4. Matrix dot product. If $C=A\cdot B$ , A’s column(n) need to be same size to B’s row(m). python: A.dot(B) or A@B
5. Matrix-Vector dot product. $C=A\cdot \upsilon$
6. Matrix-Scalar. element-wise multiplication
7. Type of Matrix
 1. square matrix. m=n. readily to add, mulitpy, rotate
 2. symmetric matrix. $M=M^T$
 3. triangular matrix. python: tril(vector) or triu(vector) lower tri or upper tri matrix
 4. Diagonal matrix. only diagonal line has value, doesnot have to be square matrix. python: diag(vector)
 5. identity matrix. Do not change vector when multiply to it. notatoin as $I^n$ python: identity(dimension)
 6. orthogonal matrix. Two vectors are orthogonal when dot product is zeor. $\upsilon \cdot \omega = 0$ or $\upsilon \cdot \omega^T = 0$ . which means the project of $\upsilon$ to $\omega$ is zero. An orthogonal matrix is a matrix which $Q^T \cdot Q = I$
8. Matrix Operation
 1. Transpose. $A^T$ number of rows and columns filpped. python: A.T
 2. Inverse. $A^{-1}$ where $AA^{-1}=I^n$ python: inv(A)
 3. Trace. $tr(A)$ the sum of the values on the main diagonal of matrix. python: trace(A)
 4. Determinant. a square matrix is a scalar representation of the volume of the matrix. It tell the matrix is invertable. $det(A)$ or $|A|$ . python: det(A) .
 5. Rank. Number of linear indepent row or column(which is less). The number of dimesions spanned by all vectors in the matrix. python: rank(A)
9. Sparse matrix
 1. sparsity score = $\frac{count of non-zero elements}{total elements}$
 2. example: word2vector
 3. space and time complexity
 4. Data and preperation
 1. record count of activity: match movie, listen a song, buy a product. It usually be encoded as : one hot, count encoding, TF-IDF
 5. Area: NLP, Recomand system, Computer vision with lots of black pixel.
 6. Solution to represent sparse matrix. reference
 1. Dictionary of keys: (row, column)-pairs to the value of the elements.
 2. List of Lists: stores one list per row, with each entry containing the column index and the value.
 3. Coordinate List: a list of (row, column, value) tuples.
 4. Compressed Sparse Row: three (one-dimensional) arrays (A, IA, JA).
 5. Compressed Sparse Column: same as SCR
 7. example
 1. covert to sparse matrix python: csr_matrix(dense_matrix)
 2. covert to dense matrix python: sparse_matrix.todense()
 3. sparsity = 1.0 – count_nonzero(A) / A.size
10. Tensor
 1. multidimensional array.
 2. algriothm is similar to matrix
 3. dot product: python: tensordot()

Factorization

Matrix Decompositions
1. LU Decomposition
 1. square matrix
 2. $A = L\cdot U \cdot P$ , L is lower triangle matrix, U is upper triangle matrix. P matrix is used to permute the result or return result to the orignal order.
 3. python: lu(square_matrix)
2. QR Decomposition
 1. n*m matrix
 2. $A = Q \cdot R$ where Q a matrix with the size mm, and R is an upper triangle matrix with the size mn.
 3. python: qr(matrix)
3. Cholesky Decomposition
 1. square symmtric matrix where values are greater than zero
 2. $A = L\cdot L^T=U\cdot U^T$ , L is lower triangle matrix, U is upper triangle matrix.
 3. twice faster than LU decomposition.
 4. python: cholesky(matrix)
4. EigenDecomposition
 1. eigenvector: $A\cdot \upsilon = \lambda\cdot \upsilon$ , $A$ is matrix we want to decomposite, $\upsilon$ is eigenvector, $\lambda$ is eigenvalue(scalar)
 2. a matrix could have one eigenvector and eigenvalue for each dimension. So the matrix $A$ can be shown as prodcut of eigenvalues and eigenvectors. $A = Q \cdot \Lambda \cdot Q^T$ where Q is the matrix of eigenvectors, $\Lambda$ is the matrix of eigenvalue. This equotion also mean if we know eigenvalues and eigenvectors we can construct the orignal matrix.
 3. python: eig(matrix)
5. SVD(singluar value decomposition)
 1. $A = U\cdot \sum \cdot V^T$ , where A is m*n, U is m*m matrix, $\sum$ is m*m diagonal matrix also known as singluar value, $V^T$ is n*n matrix.
 2. python: svd(matrix)
 3. reduce dimension
 1. select top largest singluar values in $\sum$
 2. $B = U\cdot \sumk \cdot Vk^T$ , where column select from $\sum$ , row selected from $V^T$ , B is approximate of the orignal matrix A.
 3. `python: TruncatedSVD(n_components=2)

Stats

Multivari stats
1. variance: $\sigma^2 = \frac{1}{n-1} * \sum{i=1}^{n}(xi-\mu)^2$ , python: var(vector, ddof=1)
2. standard deviation: $s = \sqrt{\sigma^2}$ , python:std(M, ddof=1, axis=0)
3. covariance: $cov(x,y) = \frac{1}{n}\sum{i=1}^{n}(xi-\bar{x})(y_i-\bar{y})$ , python: cov(x,y)[0,1]
4. coralation: $cor(x,y) = \frac{cov(x,y)}{sx*sy}$ , normorlized to the value between -1 to 1. python: corrcoef(x,y)[0,1]
5. PCA
 1. project high dimensions to subdimesnion
 2. steps:
 1. $M = mean(A)$
 2. $C = A-M$
 3. $V = cov(C)$
 4. $values,vector = eig(V)$
 5. $B = select(values,vectors)$ , which order by eigenvalue
 3. scikit learn
 pca = PCA(2) # get two components pca.fit(A) print(pca.componnets_) # values print(pca.explained_variance_) # vectors B = pca.transform(A) # transform to new matrix
- Linear Regression
1. $y = X \cdot b$ , where b is coeffcient and unkown
2. linear least squares( similar to MSE) $||X\cdot b - y|| = \sum{i=1}^{m}\sum{j=1}^{n}X{i,j}\cdot (bj - y_i)^2$ , then $b = (X^T\cdot X)^{-1} \cdot X^T \cdot y$ . Issue: very slow
3. MSE with SDG