In order to meet the growing power demand and concerns about climate change, the integrated Microgrid and Distributed Energy Systems (MDES) with the power system has become increasingly prevalent. While linking MDES to large-scale grids can enhance the economic efficiency of power grid, it also introduces technical challenges by changing thedamping performance of grid. This alternation may give rise to Low Frequency Oscillations (LFOs), posing a threat to grid stability in certain situations. So, it becomes crucial to detect such modes of LFOs online to implement corrective measures to maintain system stability. The development of Wide Area Monitoring Systems (WAMSs) based on Phasor Measurement Units (PMUs) has made it possible to monitor the small signal rotor angle stability online by accessing real-time power system measurements at the control center.The primary objective of the current thesis is to deal with a few issues related to the online mode estimation of LFOs for ringdown and ambient data in power systems. For mode estimation in the case of oscillatory ringdown data, significant work has been done by researchers in the recent past. Different mode estimation techniques have been developed to characterize these modes based on their dominance in the signal, which requires prior knowledge about the signal&rsquos Model Order (MO). So, the key contribution of this work is to develop a robust MO estimation technique, which will work efficiently in all ranges of signal and noise levels. So, in this proposed work, an unsupervised sequential K-Mean++ clustering technique is implemented for the estimation of model order of low-frequency PMU data by separating the significant eigenvalues of Autocorrelation Matrix (ACM) iteratively into signal and noise subspace. This two-layer K-Means++ algorithm promises a valid estimation of model order so that the insignificant eigenvalues are hindered from getting clustered into the signal subspace. The estimated MO is considered for mode estimation through modified Total Least Square Estimation of Signal Parameters via Rotational Invariance Techniques (TLS-ESPRIT) for the estimation of modes. To validate the efficiency and robustness of the proposed method, a comparative study is carried out with the other recently developed techniques.In order to further enhance the superiority mode estimation techniques in noisy environments and to reduce the computational complexity, a unified two-stage model characterisation technique is discussed for accurate low frequency mode estimation. As the measured signal usually contains noise, initially a combination of Tunable Q-factor Wavelet Transform (TQWT) and basis pursuit denoising technique is implemented to eliminate the interferences and improve the identification results. Subsequently, a novel sequential partitioning technique is considered to find out the MO. The proposed MO estimation technique utilizes the Convex Combination of Eigenvalues Weightage (CCEW) to scale the dominance of eigenvalues in the trace of ACM, thereafter an Advanced Medoid-based Partitioning (AMP) is implemented in a sequential manner to segregate these modes into two categories, i.e the signal subspace and noise subspace. So, this improved MO estimation approach enhances the noise resistance and reduces the computation time. As the eigenvalues of ACM are highly affected by bad measurements and outliers, it is observed that MO estimated by the above techniques are severely affected, making the mode estimation techniques inefficient and harder to automate in real-time. So, to overcome the aforementioned limitations, this part of the thesis proposes a robust mode estimation technique that precisely detects the signal low frequency modes even in the presence of high variance noise and outliers. So, in this proposed work, an Annihilating Filter-based Low-rank Hankel Matrix (ALOHA) technique is implemented to obtain the rank deficient Hankel matrix to nullify the existence of noise and outliers in PMU signal, thereafter approximated low rank Hankel matrix is considered for estimation of signals dominant modes. Wherein a sequential AMP is implemented for the detection of numbers of prominent low frequency modes by segregating eigenvalues of ACM into two opponents i.e the signal and noise subspace. Thereafter, the estimated MO is considered for the estimation of modes through TLS-ESPRIT.