Data Storage and Processing Systems Engineering

Major: Software Engineering
Code of subject: 7.121.01.E.013
Credits: 6.00
Department: Software
Lecturer: professor Melnyk Roman Andriyovych
Semester: 2 семестр
Mode of study: денна
Мета вивчення дисципліни: The purpose of studying the academic discipline is for students to master the following knowledge: knowledge of software design methodologies for processing and storing data in a hardware and software environment; data processing and storage technologies: receiving and transmitting data between different environments of the information infrastructure in the modern world, including those for the recognition of data formats and the formation of reports in real time; data transformation algorithms for the selection of objects and their analysis and classification, the use of cloud technologies for analytical calculations and data storage.
Завдання: ІНТ. The ability to effectively solve specialized tasks and practical problems of an innovative nature during professional activities related to all aspects of software development from the initial stages of specification creation to system support after commissioning. ФКС1.1. Possessing in-depth knowledge of information data models and systems, the ability to create innovative software for storage, mining and processing of large volumes of data.
Learning outcomes: РНС1.1. Be able to apply technologies of data processing, storage and transferring between different modern information infrastructure mediums including those for data format recognition and real-time report generation. РНС1.2. Know data transformation algorithms for object extraction, analysis and classification and be able to use cloud technologies for analytical calculations and data storage.
Required prior and related subjects: Prerequisites: Research Methods and Tools in Software Engineering Corequisites: Master’s Thesis Preparation and Defence
Summary of the subject: Discipline is necessary to obtain general and professional competencies in the field of algorithms and methods of data processing and storage. The discipline considers the main stages: analysis and classification, methods of storage and retrieval. The principles, methods and algorithms for calculating data characteristics, classifications, methods of cluster analysis for data retrieval are presented. Methodologies for designing software for data processing and storage in a hardware-software environment; technologies of data processing and storage: reception and transmission of data between different environments of information infrastructure in the modern world, including in recognition of data formats and formation of reports in real time; data conversion algorithms for object selection and analysis and classification, use of cloud technologies for analytical calculations.
Опис: Relational data models and DBMS implementing them. Requests for different types of data sources. Inverted indices. Mathematical apparatus of description. Operations on data and their implementation. Data analysis Data visualization. Non-relational data models. Associations between records. Examples of non-relational databases. MongoDB as a common database. Queries to non-relational databases. Searching for sources by metadata. Keyword search operations. Data processing in Mongo DB. Different data sources, queries, keywords, indexes. Integration of data from different sources. ETL technology. EII technology. EAI technology. ECM technology. WEB data integration. Integration of information systems on based on Web services. Protocols of services. Architecture of OLAP systems. Processing of data from various sources in modern technologies and in various programming languages. Data space model. Multi-level structures. Examples of data management systems based on MySql for direct and program use. Procedural and object approach. Programming tools in DBMS. Accelerated data search algorithms. Cluster analysis. An example of data storage in MS SQLServer. Query language, procedures. Network transmission mechanisms. Distributed data storage. Problems, advantages, disadvantages on the examples of VISA, Mastercard, etc. Data storage networks based on GRID examples, possible architecture. Data storage in the clouds using the Azure system as an example. Software interface. Comparison with other technologies. Data storage in the clouds using the example of the AWS system by Amazon. Access mechanisms. Data dimensionality. An example of data storage in the VNS system. Data types, entities, attributes. System performance.
Assessment methods and criteria: 1. Examination control (written and spoken components). 2. Questioning in classes. 3. Tests. 4. Defense of control exercises performed at home
Критерії оцінювання результатів навчання: Current control (45%): written reports on laboratory work (30%, 3 labs 10% each), tests on classes (15%) Final control (55%, exam), testing (50%), oral examination - (5%)
Recommended books: 1. Дейт К.Дж. Введение в системы баз данных, 8-е издание.: Пер. с англ. . – М.: Изд. Дом «Вильямс», 2017. – 1328 с. 2. Коннолли Т., Бегг К. Базы данных. Проектирование, реализация и сопровождение. Теория и практика, 3-е изд. – М.: Изд. Дом «Вильямс», 2017. – 1440 с. 3. Ramez Elmasri, Shamkant B. Navathe. FUNDAMENTALS OF Database Systems SEVENTH EDITION [Електронний ресурс] – Режим доступу: https://www.auhd.site/upfiles/elibrary/Azal2020-01-22-12-28-11-76901.pdf 4. Distributed Database Management Systems [Електронний ресурс] – Режим доступу:https://cs.uwaterloo.ca/~tozsu/courses/cs856/F02/lecture-1-ho.pdf 5. Distributed DBMS Tutorial [Електронний ресурс] – Режим доступу: https://www.tutorialspoint.com/distributed_dbms/index.htm