Home/ Majors directory/Cyber-Physical Systems/Technologies of Machine learning in Cyber-Physical Systems
Technologies of Machine learning in Cyber-Physical Systems
Major: Cyber-Physical Systems
Code of subject: 7.123.04.O.010
Credits: 4.00
Department: Electronic Computing Machines
Lecturer: PhD, associate professor Botchkaryov Oleksy Yuriyovich
Semester: 2 семестр
Mode of study: денна
Завдання: General competences:
– basic knowledge of fundamental sciences, to the extent necessary for mastering general professional disciplines;
- basic knowledge in the field of computer engineering (computer logic, programming theory, electrical engineering) is necessary for mastering professionally oriented disciplines.
– the ability to analyze and synthesize;
– ability to apply knowledge in practice;
- the ability to search and analyze information from various sources;
- the ability to solve tasks and make appropriate decisions;
- creativity, ability to system thinking.
Professional competences:
- basic knowledge of technical characteristics, design features, purpose and rules of operation of computer systems, networks and equipment;
- knowledge of modern technological processes and automation systems for technological preparation of production;
– ability to apply and integrate knowledge and understanding of other engineering disciplines;
- the ability to investigate a problem and identify constraints, including technical and sustainability issues, as well as environmental impact and life safety issues.
Learning outcomes: Learning outcomes according to the educational program:
ZN1 Know and understand the scientific and mathematical principles underlying the functioning of computer tools, systems and networks.
ZN5 Know and understand design methodologies, relevant regulatory documents, current standards and technical conditions.
UM2 To be able to apply knowledge and understanding to solve problems of analysis and synthesis characteristic of the chosen specialization; to be able to think systematically and apply creative abilities to the formation of fundamentally new ideas.
UM5 Be able to search for information in various sources to solve computer engineering problems; to be able to work effectively both individually and as part of a team.
The results of studying the discipline:
1. to know the principles of building autonomous intelligent cyber-physical systems and be able to apply them when building modern cyber-physical systems;
2. to know the general principles of functioning of autonomous intelligent cyber-physical systems;
3. to know the methods of machine learning and how to use them in the work of autonomous intelligent cyber-physical systems;
4. to have practical skills in creating and adjusting the operation of autonomous intelligent cyber-physical systems;
5. to be able to research, design and implement autonomous intelligent cyber-physical systems based on machine learning methods, principles of adaptive management and principles of self-organization;
6. to have practical skills in working with machine learning methods, in particular with reinforcement learning methods.
Required prior and related subjects: Prerequisites: Computer logics, Algorithms and calculation methods.
Corequisites: Technologies of artificial intelligence in computer and cyberphysical systems.
Summary of the subject: The discipline "Technologies of machine learning in cyber-physical systems" aims to develop in students a systematized understanding of the main provisions and principles of the theory of machine learning and approaches to the practical implementation of autonomous intelligent cyber-physical systems. As a result of mastering the study material of the discipline, students should understand conceptual issues and the multifaceted nature of the problem of developing and organizing the work of autonomous intelligent cyber-physical systems, know the problems and methods associated with the application of machine learning technologies to solve specific problems of building autonomous distributed cyber-physical systems, be able to create, configure and adjust the operation of autonomous intelligent cyber-physical systems. To master this discipline, knowledge of the following disciplines is necessary: "Computer logic", "Algorithms and calculation methods".
Опис: 1. Autonomous intelligent CFS based on machine learning technologies. Use of artificial intelligence and machine learning technologies in KFS. The concept of artificial intelligence. The main directions of research in the field of artificial intelligence. Basic concepts of artificial intelligence. Use of artificial intelligence methods to build intelligent systems. Autonomous intelligent CFS. The concept of an intelligent system. The problem of developing intelligent systems. Decision-making in conditions of uncertainty.
2. Learning automata. Mathematical modeling of simple forms of purposeful behavior. Stationary random environment. Learning automata. Asymptotically optimal sequences of learning symmetric automata. A random environment with switching states. Behavior of learning automata in random environments with state transitions. A cascade of two machines with linear tactics. Stochastic automata with variable structure. Collective behavior of learning automata.
3. Training with reinforcement (Reinforcement Learning). Machine learning. Classification of machine learning methods. Reinforcement learning. Formulation of a generalized problem of learning with reinforcement. Classification of learning tasks with reinforcement. One-factor random environment (Multi-armed bandit problem). Action-value method. Generalized form of methods of weighted evaluation of actions. The method of exponential (weighted by age) averaging. Method of normalized exponential function (softmax action selection). A method based on stochastic gradient ascent. Upper-Confidence-Bound action selection method. Comparison of effectiveness of reinforcement learning methods in a univariate random environment. One-factor random environment with contextual dependence (Contextual bandit). Markov Decision Process. Finding the optimal strategy for a known MDP model. Methods of learning with reinforcement based on temporal differences (Temporal difference learning). Adaptive Heuristic Critic method. SARSA reinforcement learning method. Q-learning reinforcement learning method.
4. Autonomous CFS based on intelligent agents. Concept of intelligent agent architecture. Agent architectures based on models of logical thinking. Architecture of integral subordination (R. Brooks). Comparison of cognitive and reactive architectures of intelligent agents. Combined architectures of intelligent agents. Development directions of architectures of intelligent agents.
5. Machine learning in autonomous multi-agent CFS. The concept of a multi-agent system. Conceptual model of a multi-agent system. Algorithmic support of multi-agent systems. Mechanisms of coordination of collective behavior of intellectual agents. Reinforcement learning in multi-agent systems. The problem of recognizing the state and interpreting the response of the environment. Classification of tasks of collective learning with reinforcement. Models of collective learning. Informational interaction of agents in the learning process. Models of information interaction of intelligent agents. Learning the distribution of responsibilities in a multi-agent system (learning organizational roles).
6. Using the principles of self-organization when building autonomous intellectual CFS. The concept of self-organization. Ways of evaluating the process of self-organization. Assessment of the self-organization process based on Shannon entropy. Heinz von Forster's model of the self-organization process. Self-organization in autonomous decentralized systems. Models of structural self-organization.
Assessment methods and criteria: Written reports on laboratory work, the verbal questioning (40%). Final assessment (60 %, control method, exam): written-verbal form (60%).
Критерії оцінювання результатів навчання: The semester grade is issued on the condition that the student completes the study plan. The semester grade is formed from the results of current monitoring of laboratory work and semester testing. The result of the semester testing is the product of the result of the semester test in the virtual learning environment and the coefficient of the lecture tests in the virtual learning environment. Maximum score in points - 100. Current control - 40. Examination control: written component - 50, verbal component - 10.
Порядок та критерії виставляння балів та оцінок: 100–88 points – (“excellent”) is awarded for a high level of knowledge (some inaccuracies are allowed) of the educational material of the component contained in the main and additional recommended literary sources, the ability to analyze the phenomena being studied in their interrelationship and development, clearly, succinctly, logically, consistently answer the questions, the ability to apply theoretical provisions when solving practical problems; 87–71 points – (“good”) is awarded for a generally correct understanding of the educational material of the component, including calculations, reasoned answers to the questions posed, which, however, contain certain (insignificant) shortcomings, for the ability to apply theoretical provisions when solving practical tasks; 70 – 50 points – (“satisfactory”) awarded for weak knowledge of the component’s educational material, inaccurate or poorly reasoned answers, with a violation of the sequence of presentation, for weak application of theoretical provisions when solving practical problems; 49-26 points - ("not certified" with the possibility of retaking the semester control) is awarded for ignorance of a significant part of the educational material of the component, significant errors in answering questions, inability to apply theoretical provisions when solving practical problems; 25-00 points - ("unsatisfactory" with mandatory re-study) is awarded for ignorance of a significant part of the educational material of the component, significant errors in answering questions, inability to navigate when solving practical problems, ignorance of the main fundamental provisions.
Recommended books: 1. Stuart Russell, Peter Norvig, Artificial Intelligence: A Modern Approach, 4th edition, Pearson, 2020. - 1136 p.
2. Richard S. Sutton, Andrew G. Barto, Reinforcement Learning: An Introduction, A Bradford Book, 2 ed., MIT Press, Cambridge, MA, 2018. - 322 p.
3. Multiagent Systems, by Gerhard Weiss (Editor), 2nd edition, The MIT Press, 2013. - 920 p.
4. Adrian A. Hopgood, Intelligent Systems for Engineers and Scientists: A Practical Guide to Artificial Intelligence, 4th ed., CRC Press, 2021. – 515 p.
5. Narendra, K. and Thathachar, M. A. L., Learning Automata: An Introduction, 2nd ed., Dover Publications, 2013. - 496 p.
6. K. Najim, A.S. Poznyak, Learning Automata: Theory and Applications, Elsevier, 2014. – 236 p.
7. Chowdhary, Chiranji Lal, Intelligent systems: advances in biometric systems, soft computing, image processing, and data analytics, Apple Academic Press, 2020. – 320 p.
8. Maxim Lapan, Deep Reinforcement Learning Hands-On, 2nd edition, Packt Publishing, 2020. - 798 p.
9. Richard E. Neapolitan, Xia Jiang, Artificial Intelligence: With an Introduction to Machine Learning, Chapman and Hall, 2018. - 480 p.
10. Laurence Moroney, AI and Machine Learning for Coders: A Programmer's Guide to Artificial Intelligence, O'Reilly Media, 2020. - 390 p.
11. Leon Reznik, Intelligent Security Systems: How Artificial Intelligence, Machine Learning and Data Science Work For and Against Computer Security, Wiley-IEEE Press, 2021. – 371 p.
12. Artificial Intelligence-based Internet of Things Systems, Souvik Pal, Debashis De, Rajkumar Buyya (eds.), Springer, 2022. – 513 p.
Уніфікований додаток: Lviv Polytechnic National University ensures the realization of the right of persons with disabilities to obtain higher education. Inclusive educational services are provided by the Service of accessibility to learning opportunities "Without restrictions", the purpose of which is to provide permanent individual support for the educational process of students with disabilities and chronic diseases. An important tool for the implementation of the inclusive educational policy at the University is the Program for improving the qualifications of scientific and pedagogical workers and educational and support staff in the field of social inclusion and inclusive education. Contact at:
St. Karpinsky, 2/4, 1st floor, room 112
E-mail: nolimits@lpnu.ua
Websites: https://lpnu.ua/nolimits https://lpnu.ua/integration
Академічна доброчесність: The policy regarding the academic integrity of the participants of the educational process is formed on the basis of compliance with the principles of academic integrity, taking into account the norms "Regulations on academic integrity at the Lviv Polytechnic National University" (approved by the academic council of the university on June 20, 2017, protocol No. 35).