In the world of control systems, Linear Quadratic Regulator (LQR) controllers have long been a staple for managing linear systems due to their simplicity. However, the rise of machine learning, particularly neural networks, offers an intriguing alternative to traditional control methods. In this semestral project, we aimed to approximate the LQR state feedback controller with a neural network through a data-driven approach.
LQR Controllers
LQR controllers are primarily designed for linear systems and require precise mathematical models. They operate with fixed parameters and are non-adaptive, making them suitable for systems where the dynamics are well understood and remain consistent.
Neural Networks as Controllers
Neural networks, on the other hand, learn from data and can manage model inaccuracies, making them ideal for systems with nonlinearities or where the model may not be precisely known.
Closed-Loop System
The closed-loop system is the core of this project. The upper part of the diagram shows the LQR controller with a feedforward component controlling the state-space model. The controlled system, described by a state-space model, involves two states, one input, and one output. The lower part of the diagram illustrates the neural network replicating the LQR control input. This setup includes the weighting matrix R to train the neural network for different configurations, thereby approximating various feedback gains K.
Data Preparation
The datasets for training, validation, and testing were generated by simulating the closed-loop system with the LQR controller under various system initial conditions, references, and weighting matrices:
- Training Data: Collected for multiple sets of system initial conditions, references, and weighting matrices to cover a wide range of scenarios.
- Validation Data: Used to tune hyperparameters and prevent overfitting, ensuring the model’s robustness.
- Testing Data: Evaluated to assess the performance and generalizability of the neural network.
Neural Network Architecture
The neural network architecture used in this project is a Feedforward Neural Network (FNN) with two hidden layers, comprising 12 and 14 neurons, respectively. The ReLU activation function was employed to introduce nonlinearity, enabling the network to learn complex patterns.
Training the Neural Network
The neural network was trained using the Adam optimization algorithm, known for its efficiency and effectiveness in handling large datasets. The training involved adjusting weights and biases to minimize the mean squared error between the predicted and actual control inputs.
Performance Evaluation and Results
The performance of the neural network was evaluated against the LQR controller in both continuous-time and discrete-time scenarios. The metrics used for evaluation included:
- Root Mean Squared Error (RMSE): Measures the average magnitude of the errors.
- Mean Absolute Error (MAE): Provides the average absolute difference between predicted and actual values.
- Coefficient of Determination (R²): Indicates the proportion of variance explained by the model.
Figure 3 compares the control input performance of the neural network to the LQR controller. The magenta steps represent the neural network’s control input, while the green steps represent the LQR’s control input. The high R² value (0.99998) indicates an excellent fit, showing that the neural network’s control input closely matches the LQR’s. The low RMSE (0.0034) and maximum error (0.0206) confirm minimal average and worst-case prediction errors, respectively.
This chart shows the state trajectories for both the neural network (magenta) and the LQR controller (green). The high R² values for both states (0.99991) and low RMSE values (0.0013 for the first state and 0.0027 for the second state) demonstrate the neural network’s accuracy. The maximum errors are also minimal, ensuring the precision of the neural network’s predictions.
Figure 5 illustrates the relationship between the control input and the state variables. The green color represents the LQR controller, and the magenta steps represent the neural network. The close alignment between the two sets of markers shows that the neural network effectively approximates the control law function u=f(x), generating control actions similar to those of the LQR controller.
Conclusion
In summary, this project successfully demonstrates the potential of neural networks to approximate Linear Quadratic Regulator (LQR) controllers, achieving high accuracy. This indicates that neural networks can enhance control capabilities. The neural network’s ability to generalize across different system configurations, achieved by training on diverse datasets with varying initial conditions and weighting matrices, highlights its scalability. This feature is particularly valuable for complex control systems where traditional LQR design would be cumbersome and inflexible. The neural network can also manage complex and nonlinear dynamics. The promising results pave the way for further research and application of machine learning techniques in the field of control theory.
0 Comments