Random Sparse Sampling and Equal Intervals Bregman High-Resolution Signal Reconstruction

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

Random Sparse Sampling and Equal Intervals Bregman High-Resolution Signal Reconstruction1 Guojun Qin1, Jingfang Wang2 1 – School of Mechanical Engineering, Hunan International Economics University, Changsha, China 2 – School of Electrical & Information Engineering, Hunan International Economics University, Changsha, China DOI 10.2412/mmse.74.23.960 provided by Seo4U.link

Keywords: compressed sensing, random sampling, incoherent measurement matrix, sparse sampling, Bregman reconstruction.

ABSTRACT. Compressed sensing (CS) is a new signal processing methods, signal sampling and reconstruction are processed to take full advantage of the signal sparse knowledge structure in the transform domain. It consists of three elements: the sparse matrix, incoherent measurement matrix and reconstruction algorithm. In the framework of compressed sensing theory, the sampling rate is no longer decided in the bandwidth of the signal, but it depends on the structure and content of the information in the signal. In this paper, a complex domain random observation matrix is designed and interval samples are projected to any set of random sample, ie, sparse random sampling. The signal is successfully restored by the use of Bregman algorithm. The signal is described in the transform space, and a theoretical framework is established with a new signal descriptions and processing. By making the case to ensure that the information loss, signal is sampled at much lower than the Nyquist sampling theorem requiring rate, but also the signal is completely restored in high probability. The sparse signal is simulated in sampling and reconstruction of time domain and frequency domain, and the signal length, the measured value, the signal sparse level and SNR influence are analyzed in the reconstruction error.

Introduction. Sampling theorem was the law which is followed in a sampling process, it described the relationship between the sampling frequency and the frequency spectrum of the signal. Sampling theorem was first proposed by Nyquist in 1928, who is the American telecommunications engineer, this theorem was clearly stated and formally incorporated theorem in 1948 by the founder Shannon of information theory , so it is called Shannon sampling theorem in many references. The theory dominates almost all of the signals / images acquisition, processing, storage, transmission, etc., that is: when the highest frequency of a signal is

m ,



The sampling rate as long as the 2 m is sampled to this signal, signal can be perfectly reconstructed.

This is a problem with sufficient but not necessarily, if two samples are taken to a waveform with the highest frequency, they are with necessary and sufficient, then the other one cycle of all frequency waves gather more than two samples - adequate but not necessary. As can be seen, Shannon sampling theorem is not the best, it does not make use of the structural characteristics of the signal itself – sparsity (with respect to the signal length, only a very few non-zero coefficients, and the remaining coefficients are zero), only a small part of the collected data is our actual useful work. They want to keep, but most of the rest will have to give up and thus waste a lot of storage space.

Nyquist theory codec is that encoding end of the signal is sampled first, and then the samples were all converted and encoded for amplitude and position of one important factor, and finally the coded value were stored or transmitted; decoding of the signal is only the reverse process of encoding, the signal is restored by the de-compression, inverse transform.

© 2017 The Authors. Published by Magnolithe GmbH. This is an open access article under the CC BY-NC-ND license http://creativecommons.org/licenses/by-nc-nd/4.0/

MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

There are two shortcomings in Nyquist theory: (1) Since the sampling rate of the signal can not be lower two times than the signal bandwidth (frequency difference between the highest and the lowest signal frequency), which makes the hardware system is facing great pressure in the sampling rate; (2) In the compression encoding process, in order to reduce the cost of storage, processing and transmission, of a large number of conversion calculated small coefficients are discarded, it results in a waste of data computing and memory resources. This high-speed sampling re-compressing process wastes a lot of sampling resources, so it naturally raises a question: Can other transform space is used to describe signal? A new theoretical framework is created for describing and processing signal, so that the information is not loss case, the required sampling rate is used with far lower than the Shannon sampling theorem to sample signal, while the signal can be fully restored? Whether can the signal sample be transformed into the sampling information?

Signal sampling, compression encoding occur in one step in compressed sensing (Compressive sensing / sampling, CS) [1, 2], sparse signals are used, it is far below the Nyquist sampling rate of the signal, non-adaptive measurement is encoded. Measured values are not signal itself, but it is projection value from the high dimension to low-dimensional space, from a mathematical point view, each measurement value is the combination function of on each sample signal under the traditional theory, a measured value already contains the small amount of information in all the sample signal.

Decoding process is not simple reverse process of encoding, but under the inverse thought in the blind source separation, the signal sparse decomposition is used in conventional reconstruction methods to achieve accurate reconstruction of a signal in the sense of probability or do approximate weight structure under certain error. The required number of measurements for decoding is much smaller than the number of samples of Nyquist theory. Correspondence between the Nyquist sampling theory and CS as shown in Table 1 [3], [4].

Table 1. Correspondence between Nyquist sampling and CS Theory.

Nyquist sampling

CS Theory

Prerequisites

Signal bandwidth

Sparsity signal

Sampling Method

Local uniform intervals

Global,Randomly

Key point

Nyquist sampling rule

Non-correlation measurement matrix

Means of the reconstructed signal

Fourier transform

Nonlinear optimization

Complexity

Sampling

Computing

If you want to capture a small part of the data and expect to "decompress" a lot of information from these few data, it is necessary to ensure that: 1) The amount of collected data contains global information of the original signal; 2) There is an algorithm which the original information can be restored from a small amount of data.

In this study, a compressed sensing sparse random sampling method is researched with highresolution signal reconstruction. A complex domain measurement matrix is devised in the method, the non-uniform sampling is limit from the sampling frequency, there are frequency advantage of high resolution and anti-aliasing [5], at a low sampling frequency, the separate Bregman method is used to reduce spectral noise which is caused by non-uniform sampling, the even spaced signals are reconstructed with the high-resolution [6, 7].

MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

Compressed sensing principle Compressed Sensing statements. The main idea of compressive Sensing (Compressive Sensing CS) theory is that: Suppose the coefficients of a signal x with length N are sparse (that is, only a small number of non-zero coefficient) on orthogonal basis or on a tight frame  , if the coefficients are projected to another observation group  : M  N ， M  N , which is not related to the transform group  , the observation set y:M 1 is obtained. Then the signal x can rely on these observations to solve an optimization problem and accurate recovery. CS theory is the theoretical framework with a new sampling while achieving the compression purpose, its compressive sampling procedure is shown in Figure 1.

Fig. 1. Compression sampling process.

First, if the signal x  R N is compressible on an orthogonal base or tight frame  , the transform coefficients   T x are obtained,  is equivalent x or approximation sparse representation; The second step, a smooth measurement matrix  is designed, which is irrelevant to the transform base  with M  N dimension, the observation x is projected to M-dimensional space to give the set of observations y  x , the sampling process is compressed, i.e. drawing sample [8]. Finally, the optimization problem solving xˆ approach x exactly or approximately.

When the observations with noise z,

y  x +z (1)

It can be converted for the sake:

min || T x ||1 x

or xˆ  arg min x

s.t. || y  x || 2   (2)

1 || y  x || 2  ||  T x ||1 (3) 2

Recovery signals separable Bregman iterative algorithm.

Question (3) solving is first converted to sparse vector (4) solving, A    , then MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

ˆ  arg min 

 2

|| y  A || 22  ||  ||1 （4）

Specific steps of Bregman algorithm are as follows [9, 10]: T 1 T Step 1: to calculate B  (A A  I N ) , I N is the N-order unit matrix, F  A y ; b0 , d 0 are Ndimensional zero vector;

Step 2:  ( 10) is given, the iteration terminated conditions  ( 0.001) ，the number of iterations n  1; Step 3: to calculate

 n  B( F  d n1  bn1 ) ,

d n  sign( n  bn1 ) max(|  n  bn1 | 1,0) ， bn  bn1   n  d n Step 4: if

;

||  n   n1 ||  ，

  n ; n  n  1 , go to step 3; otherwise, the iteration stops, ˆ

Step 5: xˆ  ˆ .

Sparse random sampling design Signal sparse representation. Transform based of adaptive signal is  , the signal expression should be sparse at the base[11]. Transform coefficient vector of signal X is under the transform base 1 p p

 , if 0 <P <2 and R> 0, these coefficients satisfy: ||  || p  ( |  i | )  R , the coefficient vector is i

sparse [12]. If the support domain {i :  i  0} of the transform coefficients i  X , i  is equal or less K, namely there is K nonzero entries only in   R N . The number K of non-zero entries reflect the signal inherent freedom. Or sparseness is a measure of non-zero coefficients, and it constitutes a number of scales of the signal component. Typically, sparse representation of the signal can be measured by the 0-norm of the representing vector. A vector 0-norm is the number of nonzero elements in the vector. Fourier transform is our common one [13], [14].

Irrelevant and Isometric properties. The adaptive M  N -dimensional measurement matrix  is designed which is not related to transformation base  . Observation matrix  design goal is to restore the original signal as little as possible from the observation values. In the specific design, it is need to consider the following two aspects: 1) the relationship between the observation matrix  and the base matrix  ; 2) the relationship between the matrix A   and K- sparse signals  .

First, the observation matrix  and the base matrix  have incoherence. Coherent between observation matrix  and the base matrix  is defined as formula (5):

 (, )  N max 1k m | k , j | (5) 1 j  N

MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

The degree of coherence  gives the maximum coherence between any two vectors in  and  .

When  and  contain coherent vectors, their coherence degree is coherence. The compressed sampling of the signal makes for each observation contains the different information of the original signal as much as possible, which requires orthogonal between the vectors of  and  , that degree of coherence  is as small as possible, which is the reason of incoherence between the measurement matrix and base matrix [15]. Just to satisfy the following formula, signal can be reconstructed high probability [16].

M  C 2 (, ) K log N (6)

Second, the relationship between the matrix A   and the K- sparse signals is about Restricted Isometry Property (RIP) [17], the matrix "equidistance constant": any K = 1, 2, ⋯, the matrix A isometric constant  K is defined to satisfy the minimum value of the following formula (7), which  is optionally K- sparse vector: (1   K ) ||  || 2 || A || 2  (1   K ) ||  || 2 2

2

2

(7)

If  K <1, Matrix A is called to satisfy the K-order RIP, matrix A is approximately to ensure in this time that the Euclidean distance of K- sparse signal  remains unchanged, which means that  is impossible in null space of A (otherwise there will be infinitely many solutions for  ). In practice, random matrix is commonly used as observation, common are Gaussian measurement matrix, binary measurement matrix, Fourier observation matrix and irrelevant observation matrix . Random observations provide an effective way to achieve the compressed samples Low-rate signal sampling and measurement matrix design. In fact, the design of the observation partis is to design an efficient observation matrix, that is to design efficient observation (ie, sampling) agreement of useful information to capture a sparse signals, whereby the signal is compressed into a small number of sparse data. The agreement is non-adaptive, it requires only a small amount of link between the fixed waveform and the original signal, the fixed waveform is irrelevant to signal compact represented base. In addition, the observation process is independent of the signal itself. the reconstructed signal can be collected by optimization methods in a small number of observations. Measurement matrix is determined by the measured waveform and sampling methods. Measured waveform generally includes a Gaussian random waveform with independent and identically distribution, Bonu Li random waveform and orthogonal functions; sampling methods include a uniform sampling, random sampling and jitter sampling.

Sampling interval is [0, T], M points were collected randomly in this interval, rand (0,1) ti  rand (0,1)T , i  1,2,, M , is random point in interval (0,1]; ~ x  [ x(t ), x(t ),, x(t )]T , y  ~ x  R M , N-dimensional complex frequency domain vector 1

2

M

  C , M  N is reconstructed at equal intervals.

N

Random measurement matrix  is designed:

(m, n) 

1 N /2 l t  exp( 2i ( m  n)), m  1,2,, M ; n  1,2,, N (8) N l  N / 21 N Ts MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

Ts is the equivalent sampling interval in the time domain which is used to reconstruct N points uniformly, 

is Fourier base,  j (tn )  N 1/ 2ei 2jt n , j, n  1,2,, N , t m belongs to domain

collections of the reconstructed signal including a random collection. Such design of  and  meet incoherence in formula (5) and Restricted Isometry Property (RIP) in formula (7). A   ， This random extraction makes each observation with random uncorrelated characteristics. The irrelevant feature between Stochastic Observation Matrix is a sufficient condition for proper signal recovery, if observation matrix is highly irrelevant with the signal, it can ensure to restore the effective signal.

N points x*  [ x(Ts ), x(2Ts ),, x( NTs )]T are sampled in the sampling space Ts in interval [0, T], then  is a mapping:  : x*  R N  ~ x  R M , it is proven below by discrete Fourier transform formula and inverse Fourier transform formula:

N  n 1

l tm (  n))] N Ts

(m, n) x* (n) 

1 N

N /2 N  [  x* (n) exp( 2i l   N / 21 n 1



1 N

N /2 N  [  x* (n) exp( 2i l   N / 21 n 1



1 N

N /2  l   N / 21

X (l ) exp( 2i

lt l n)] exp( 2i m ) (9) N NTs

lt m t )~ x ( m )  x(tm ) NTs Ts

The compressed sensing harmonic detection is to find a way of the reversed mapping  : F : R M  C N (frequency domain), or further mapping: F : R M  R N (time domain).

Compressed sensing sampling signal recovery implementation steps 1) In the given interval (0, T) of the time domain, the maximum number of samples M is set based on the maximum frequency of reference signal contained in this interval, this point sequence is the observation vector; the range of reconstruction points N (> M) is determined to decide the equivalent sampling interval Ts; 2) N points are reconstructed at equal interval Ts in this section, thereby a frequency domain 1 resolution f  Hz is given; NTs 3) From formula (8), M  N -dimension observation matrix  is designed, N  N - dimension inverse Fourier transform matrix  is designed; 4) M points ~ x  [ x(t1 ), x(t2 ),, x(tM )]T are randomly sampled n the given interval (0, T); 5) Bregman algorithm is used to reconstruct N-points ˆ in complex frequency domain,

ˆ  arg min 



2

|| y  A || 22  ||  ||1 , y= ~ x;

6) Inverse Fourier transform is used to obtain N-point signals xˆ  real (ˆ ) at equal intervals in time domain.

7) Real-time processing of the next frame, go to step 4.

Experimental testing and evaluation MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

Experimental select signal function is

x(t )  2 cos(40t 

 6

)  cos(140t 

2  )  cos(300t  ) (10) 5 4

The highest frequency of the signal is f max  150Hz , M = 128 points are sampled randomly within 1 second, that is, its random equivalent sampling frequency is f s  128Hz  f max , random equivalent sampling frequency is much less than twice the highest frequency of signal, it does not meet the Nyquist sampling theorem. In terms of such a uniform sampling frequency, then there must be a Fourier transform spectrum aliasing and disclosure, it is impossible to detect the signal harmonics. M = 256 points are sampled, and N = 4 128  512 points are reconstructed in frequency domain by this method, its resolution is 1 Hz, the signal contained frequency in the original signal is same as the measured frequency of the reconstructed signal, they were 20,70,150. The original corresponding amplitudes were 2, 1, 0.5, while the results of our method were 1.9823,0.9798,0.4830; the original corresponding phase were 30 ℃, 72 ℃, 45 ℃, and the results of method were 29.9256 ℃, 71.9750 ℃, 45.2006 ℃; the detection results are shown in Figure 2. Each harmonic frequency, amplitude and initial phase of the true values and the detected values were plotted on the same graph, the result is very accurate.

Fig. 2. Compressed sensing sparse sampling and signal reconstruction n The above picture of figure 2 shows the original signal, the sampling points and the reconstruct time domain signal, the middle picture of figure 2 shows the amplitude-frequency diagram of the original signal and the reconstructed signal, the below picture of figure 2 shows the phase-frequency diagram of the original signal and the reconstructed signal. The relative error of time-domain reconstructed signal is: MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

|| x  xˆ || 2 Relative error = || x || 2 = 0. 0139.

Summary. Random sampling technique is as a non-uniform sampling method, it can effectively improve the sampling rate in sampling system. In a random sampling, the non-uniform distribution characteristics of the sampling interval is not need to collect enough samples to signal reconstruction.

In this study, the sparsity of signal is used in the Fourier transform, complex random measurement matrix  is designed, namely it is random sparse sampling; Bregman iterative algorithm is used successfully to restore signal. This method does not increase the costs on the basis of any hardware, and it is able to reconstruct the original signal by the limited random samples. Experimental results show that the frequency domain sparse signals are sampling far below the Nyquist frequency signal sampling rate, the original signal is accurately reconstructed by compressing the sensor signal reconstruction algorithms.

Compression sensing basic idea is to extract much information as little as possible from the data [1822], there is no doubt that it has a great theory and promising idea. Compressed sensing is an extension of the traditional information theory, but it is beyond the traditional compression theory, it has become a new sub-branch. When the signal has a sparse feature, compressed sensing can accurately reconstruct the source signal by a small number of observations which is much smaller than the length of signal. Compressed sensing theory is that sampling and compressed signal are combined into a single step, the signal is encoded, it breaks the traditional Nyquist sampling theorem limit in a certain extent, the burden is reduced on the hardware processing.

In the framework of the compressed sensing theory, the sampling rate is no longer determined by the bandwidth of the signal, but depending on the structure and content of the information in the signal.

It uses transform space to describe signal, a new theoretical framework is established for describing and signal processing, so that in the case to ensure that information is not lost, with far lower than signal sampling rate which the Nyquist sampling theorem requires, but also recovery signal is completed in high probability. Compared with conventional Nyquist sampling, Compression sensing reduces hardware requirements and saves resources.

Acknowledgements : This work is sponsored by the National Natural Science Foundation project (51375484) of China.

References [1] D.L. Donoho, Compressed sensing. IEEE Trans. Information Theory, 2006 , 52 (4 ) :1289-1306.

[2] Baraniuk R G. Compressive sensing. IEEE Signal Processing Magaz ine , 2007, 24(4 ) : 118 -121.

[3] Donoho D,Tsaig Y. Extensions of compressed sensing. Signal Processing,2006,86 ( 3 ):533-48.

[4] Shi G M , Liu D H , Gao D H ,Liu Z , Lin J, Wang L J . Advances in theory and application of compressed sensin. Chinese joum al of Eleetronics, 2009 , 37 (5 ) :1070-1081. (in Chinese ) [5] Shapiro H S，Silverman R A．Alias-free sampling of random noise．Journal of the Society for Industrial and Applied Mathematics, 1960，8(2)： 225-248． [6] S. Osher, Y. Mao, B. Dong, and W. Yin, Fast Linearized Bregman Iteration for Compressed Sensing and Sparse Denoising, UCLA CAM Report (08-37), 2008.

[7] W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for ℓ1-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008),pp. 143–168.

[8] E.J. Cande`s, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 489– 509, Feb. 2006.

MMSE Journal. Open Access www.mmse.xyz

Mechanics, Materials Science & Engineering, July 2017 – ISSN 2412-5954

[9] J.-F. Cai, S. Osher, and Z. Shen,, Linearized Bregman iterations for compressed sensing, Math.

Comp. 78 (2009), 1515-1536.

[10] J.-F. Cai, S. Osher, and Z. Shen, Convergence of the Linearized Bregman Iteration for ℓ1-norm Minimization, Math. Comp. 78 (2009), 2127-2136.

[11] E J Candes,J Romberg. Sparsity and incoherence in compressive sampling. Inverse Problems .

2007 ,23 (3) :969-985.

[12] E.J. Cande`s and T. Tao, Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inform. Theory, vol. 52, no. 12, pp.5406–5425, Dec. 2006.

[13] S. Chen, D. L. Donoho, M. Saunders. Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 1999, 20 (1): 33-61 [14] Candes E J and Tao T. Decoding by linear programming, IEEE Transactions on Information Theory, 2005, 51(12):4203-4215.

[15] Candes E J. The restricted isometry property and its implications for compressed sensing.

Comptes rendus del Cademie des Sciences, Serie1, 2008, 346(9-10): 589-592.

[16] Baraniuk R G, Davenport M A, DeVore R, and Wakin M B. A simple proof of the restricted isometry property for random matrices. Constructive Approximation, 2008, 28(3):253-263.

[17]Haupt J and Nowak R．A generalized restricted isometry property．Univemity of Wisconsin Madison Technical Report ECB-07-1．May 2007． [18] Liu Zhaoting, He Jin, Liu Zhong, High Resolution Frequency Estimation with Compressed Sensing, Signal Processing, 2009, 25 (8),1252-1256 [19] HE Ya-peng, LI Hong-tao, WANG Ke-rang, ZHU Xiao-hua, Compressive Sensing Based High Resolution DOA Estimation, Journal of Astronautics, 2011, 32(6),1344-1349 [20] Jin Jian, Gu Yuan-tao, Mei Shun-liang, An Introduction to Compressive Sampling and Its Applications, Journal of Electronics & Information Technology, 2010, 32(2), 470-475 [21] L. M. Brègman, A relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming, Z. Vyčisl. Mat. i Mat. Fiz. 7 (1967), 620-631.

[22] E. Candès, J. Romberg, and T. Tao, Stable signal recovery from incomplete and inaccurate measurements,Comm. Pure Appl. Math., vol. 59, no. 8, pp. 1207–1223, Aug. 2006.

MMSE Journal. Open Access www.mmse.xyz