At present, the research on the theoretical system of multimedia image linguistics in my country is very limited. In order to further improve and develop the theoretical system of multimedia picture linguistics, this article extracts effective knowledge from the research results of educational technology, linguistics, psychology, and semiotics and establishes a teaching system based on image linguistics. Multimedia picture linguistics consists of three parts: structure, semantics, and pragmatics, corresponding to the artistic laws of multimedia pictures, the cognitive laws of multimedia pictures, and the humanization and naturalization of picture design. Therefore, the research in this article is an important part of establishing multimedia image linguistics. Starting from the research methods of multimedia technology and picture linguistics, this article introduces the theoretical framework and the relationship between the multimedia learning cognitive theory and Mayer’s multimedia learning cognitive theory and conducts in-depth discussion and demonstration. The thesis constructs a teaching system that combines picture linguistics and multimedia technology and makes it a language tool for developing and designing multimedia learning materials. Experiments have proved that the teaching system combining picture linguistics and multimedia technology proposed in this paper can provide a framework, ideas, and methods for the related research of multimedia picture linguistics and multimedia technology and make multimedia picture language a multimedia learning material design and development The language tools that can be followed can effectively improve the teaching effect under the background of information technology.
“Multimedia Picture Linguistics” is a new subject category that has emerged recently. The direct reason for its emergence is the need to carry out the construction of multimedia teaching materials widely at home and abroad at present, and the deeper reason is that in the information age. It is urgent to recognize and standardize the popular language of this era, the multimedia picture language [
“Symbol” is a meaningful medium used to convey information in the process of communication. Multimedia pictures are considered to be a language because it conforms to the definition of language; that is, it can be regarded as a symbol system and can be used to transmit knowledge information to view aesthetic hearing [
The research contributions of the thesis include the following:
The thesis establishes a teaching system based on image linguistics This article starts with the research methods of multimedia technology and picture linguistics, introduces the theoretical framework and relationship between multimedia learning cognitive theory and Mayer multimedia learning cognitive theory, and conducts in-depth discussion and demonstration This paper constructs a teaching system that combines picture linguistics and multimedia technology and makes it a language tool for developing and designing multimedia learning materials
In 2002, Professor you Sewing, a famous educational technology expert in China, put forward the concept of “multimedia picture language”. In recent years, under the guidance of professor yet, the research team of our country has been guided by professor yet. On the basis of “multimedia picture language” as the starting point, this paper makes a reasonable creation of the multimedia picture art theory and designs some rules to be followed in view of the multimedia learning material interface. In recent years, the Chinese research team has put forward the theory system of “multimedia picture linguistics”, which has further expanded its research to the level of linguistics [
From the perspective of general linguistics, this paper reveals the essential attributes and linguistic characteristics of the multimedia picture language. This paper discusses the theoretical basis for the division of multimedia picture linguistics into three parts: construction, semantics, and pragmatics, in order to verify the reasonableness of multimedia picture linguistics [
Multimedia infrastructure.
Computer network platform can provide a good environment for students, so that students can receive information both visually and audibly, thus improving their learning efficiency. We made statistics on a large number of data: visual can get 82% of the information, and auditory can get 11% of the information [
Multimedia picture is a new kind of comprehensive information picture that appears with the appearance of multimedia. First of all, the multimedia learning materials and the knowledge information exchange between people, in the multimedia picture. It also has the following several important characteristics: (1) based on the basis of plane display, multimedia images can only be based on electronic plane display, such as mobile phones and computers. In order to complete the input and output of various media information in the multimedia interactive environment, the information it needs to present and the related interactive operations also need to be carried out with the help of the display screen. (2) The picture is interactive and dynamic. With the change of time, multimedia picture will also change, and multimedia picture is not a single picture but a “picture group.” In this way, the information interaction between human and machine can be realized on the basis of controlling picture group connection and change. (3) The audio and visual media can be combined effectively to form a multimedia picture, in which the sound of the screen is synchronized with the change of the picture. Although its performance has some differences, it is an essential part of multimedia pictures. (4) There are many types of multimedia pictures. Multimedia learning materials include network course, teaching course, electronic teaching plan, teaching animation, and multimedia courseware. From the point of view of presentation form, multimedia images can be divided into tablet screen, mobile phone screen, and computer screen. Two languages meet the conditions of “language” that is shown in Table
Two languages meet the conditions of “language”.
Conditions satisfied by written language | Requirements for multimedia picture language |
---|---|
A sufficient number and type of vocabulary and vocabulary have formed a complete set of rules (such as grammar and syntax) | The basic elements and elements of a picture consisting of sufficient numbers and types |
The multimedia picture language is a symbol system. The basic elements of traditional art, such as painting, photography, animation, film, and television, have already formed the symbol system in each art category. The basic elements of multimedia picture are composed of the basic elements of the above traditional art, which are divided into four media types: picture, text, sound, and image [
The multimedia picture language can be used to convey knowledge or visual sense of hearing. The multimedia teaching material is written in the multimedia picture language, because it contains all kinds of media such as picture, text and audio, and video, which can make full use of the visual, auditory, and visual and conceptual channels of learners. So, the effect of transmitting knowledge and aesthetic sense is better than that of written language and traditional art. It must be pointed out that the use of multimedia screen language symbols for communication is necessary to pay attention which is different from the two characteristics of the character symbol, namely, “in the form of meaning” and “time-sharing presentation” as shown in the following expression.
The multimedia picture language can also be used for thinking. Multimedia picture language introduces information content into the human brain through visual and auditory visual channels and conceptual channels. These information content can be processed in the form of representation and concept in the human mind [
Experience of the tower.
In linguistics, language should generally have three characteristics: symbolism, systematizes, and venerability [
Systematic: linguistics holds that language is a system of linguistic symbols maintained by grammatical rules. The multimedia picture language is a language symbol system, its language symbols are maintained by multimedia picture art rules (grammar rules), and this maintenance should pay attention to two main points: (1) the rules of multimedia picture art regulate the symbol of visual (auditory) perception, that is, the evolution of the symbol of the basic element of the norm in the picture [
Generative: linguistics holds that although the system of linguistic symbols is made up of limited linguistic symbols and grammatical rules, it can produce an infinite variety of sentences [
Charles William Morris, a famous American pragmatism philosopher, divides semiotics into three parts based on logic and law, empiricism semantics, and pragmatism semiotics that are linguistics, semantics, and pragmatics [
If the multimedia picture language is compared to the symbolic language, the following viewpoints can be obtained: (1) the basic symbols used in the multimedia picture are all kinds of media; (2) the content of the information expressed and transmitted through the various media symbols is the teaching content; (3) the concrete situation of all kinds of media symbols in multimedia picture is the real information teaching environment. Based on the theoretical framework of semiotics and on the basis of the above viewpoints, the theoretical framework of multimedia picture linguistics can be preliminarily formed. Multimedia picture linguistics consists of three parts: picture construction, picture semantics, and picture pragmatics. Picture construction studies the structure and relationship between various media, while screen semantics studies the relationship between various media and the information they express or disseminate. Based on the theoretical framework of semiotics, and based on the above viewpoints, the theoretical framework of multimedia picture linguistics can be initially formed [
The theoretical framework of multimedia picture linguistics.
After entering the 20th century, the Swiss linguist Saussure first puts forward the concepts of “language” and “speech” in linguistics, which he called “the first bifurcation of the theory of speech activity.” That is to say, for linguistics, it is necessary to distinguish language from speech [
In the study of multimedia picture textualism, the research path of “constitution” can be adopted, and the constituent elements are analyzed in terms of the design rules of the elements and the relationship between the elements, which generalize the design object into a visual and auditory object with certain characteristics. This particular object is analyzed as a constituent element, and then the design rules of the constituent elements and the relationship between the elements are summarized as the design rules [
The basic attributes of the elements in the Multimedia Picture.
Multimedia screen elements | Basic attribute |
---|---|
Text | Color, font, size, word spacing, line spacing, layout, location, etc. |
Image | Point, line, face, composition, light, color, size, position, resolution, picture type, position |
Sound | Tone, intonation, speed, rhythm, rhythm, melody, strength, length, strength, height, sound type, etc. |
Animation, video | Color, light, composition, shooting, editing, length, size, resolution, type, position, etc. |
Interaction | Input mode, output mode, application, interactive device features, etc. |
It can be seen from the development of linguistics that although the study of linguistics is deepening, Saussure’s original idea of distinguishing language from speech has continued to this day. Based on the existing research results, the idea can be described as follows. Language is a relatively stable lexical and grammatical system which has formed a consensus. It can help people communicate with the outside world and can help people to think internally. One refers to speech (including writing), which is an act of communication, and the other is that the words spoken (including the words written) need to take into account the results of the communication [
The constituent elements are the basic unit of the picture composition. The constituent elements of multimedia picture can be summarized as four kinds of media symbols, such as picture, text, sound and image, and the interactive function of the picture, that is, graph picture, including drawing figure and still image. The intersection-interactive function is essentially a multimedia picture of the group function. The presentation of the interactive output result and the trigger of the interactive input operation are realized by the multimedia screen which contains the elements of the interactive function [
Teaching content classification diagram.
Second, this paper analyses the new visual and auditory effects derived from the coordination of various attributes in the picture elements, studies its rules, and then comes up with the corresponding picture grammar rules in turn. Although the visual and auditory effects are also related to the basic attribute components of the elements of the picture, they are not equal to the sum of these elements, but they are a new whole completely independent of these elements [
Investigation on the situation creation of mental health education in primary and secondary school.
Content | Totally agree | More agree | General | Disagree | Strongly disagree |
---|---|---|---|---|---|
You prefer the form of teaching in a multimedia environment | 26 | 23 | 13 | 2 | 1 |
You are easy to adapt to multimedia and interactive teaching | 23 | 24 | 14 | 6 | 1 |
More concerned about the teaching of multimedia and interactive materials | 11 | 16 | 19 | 9 | 2 |
Your focus is more focused with the multimedia and interactive teaching, | 12 | 18 | 26 | 8 | 3 |
The use of multimedia materials makes it easier for you to enter the appropriate situation | 18 | 21 | 24 | 15 | 4 |
After using the multimedia presentation context, your understanding of the content is easier | 20 | 26 | 13 | 3 | 1 |
The use of multimedia context makes you more memorable to the content | 16 | 17 | 18 | 4 | 1 |
From the perspective of the social function, language is nothing more than a tool. The relationship between language and speech is like the relationship between tools and the use of tools. In general linguistics, sentences are the dividing line between “language” and “speech.” The static units below sentences, including words, sounds, and meanings, belong to the study category of language, which is the internal structural model of language as a relatively stable communication tool [
In multimedia picture linguistics, according to Saussure’s idea of distinguishing language from speech, the language of multimedia picture is divided into two aspects: basic theory and theoretical application. In the view of multimedia picture linguistics, the teaching content in multimedia teaching material is expressed through multimedia picture, or in a broad sense, it is expressed by moving picture (including TV picture, animation picture, and multimedia picture). This shows that the moving picture should be equivalent to the “sentence” in general linguistics, which is the dividing line between language and speech in multimedia picture linguistics. The expressions are given as follows.
The art of multimedia picture is divided into the art of media presentation and the art of picture group, which embodies the idea of dividing the picture according to the multimedia picture. The former regards the basic elements of all kinds of media on the screen as “words” in the multimedia picture language (symbol) system. The rules of multimedia picture art are regarded as “grammar” to regulate the evolution of these words (the basic elements after evolution are called visual elements). Therefore, the multimedia picture is through these “grammar” standard “vocabulary” and has formed each kind of performance teaching content structure pattern, namely, “sentence.” Obviously, these belong to the theory of multimedia picture art, corresponding to the category of “language” in linguistics. Obviously, these contents belong to the application of basic theories and correspond to the study of “phonetics” in linguistics. The latter uses editing functions or interactive functions to group each picture according to the requirements of “disciplines”, according to the requirements of different disciplines and environments. The purpose is to accurately express the content of each discipline and achieve good teaching results.
Modern linguistics holds that linguistics is a discipline that studies language tools (i.e., symbolic systems), tool internal structure systems, and the processes and results of people using language system tools in different contexts [
Language linguistics is a kind of linguistics which takes “language” as its only object. “language” refers to the lexical and grammatical system of the combination of sound and meaning. Linguistic study of speech: the process and results of using the linguistic system tools in different contexts [
Multimedia picture linguistics draws lessons from the views of modern linguistics and semiotics, and it is divided into three parts: construction, semantics, and pragmatics [
Four elements of the teaching system.
In this teaching system, the relationship between teaching content and all kinds of media belongs to the above multimedia picture semantic research content. The medium in the teaching system refers to the physical carrier of storing and transmitting information, such as information equipment and network. The relationship and law between the media and various kinds of media as the form of information presentation in multimedia pictures belong to the research content of picture pragmatics. It focuses on the different characteristics of different media carriers such as computers and mobile devices and the matching rules of various media symbols in multimedia images [
Among them, the study on the structure of multimedia picture belongs to the category of “linguistic linguistics”, which consists of linguistic symbols and grammatical rules. In addition, according to modern linguistics, the context in linguistics of speech can be divided into small context and large context. Multimedia picture pragmatics uses the “big background” as the classroom teaching environment to study different teaching forms of picture language to meet the needs of classroom teaching strategies. According to the needs of compiling multimedia teaching materials, multimedia picture semantics extends the concept of “small situation” to the teaching content of different subjects and studies different design formats of screen languages to meet the needs of these subjects. The small context refers to the context before and after oral or written language, while the large context refers to the identity, situation, emotion, and social and cultural environment of both parties. Therefore, multimedia picture semantics and pragmatics can be regarded as the study category of “linguistics of speech.”
Picture semantics studies the relationship between all kinds of media and the teaching content it expresses or conveys, from which to sum up some understanding of the regularity of different types of teaching content and form the “picture semantic rules”. According to pedagogy, the teaching content refers to all kinds of materials and information dynamically generated by teachers and students in teaching communication activities to serve the purpose of teaching. The teaching content includes not only the contents of teaching materials [
Evaluation in the PE learning content.
Evaluation | Physical fitness | Motor skill | Theoretical knowledge | Learning attitude | Spirit of cooperation |
---|---|---|---|---|---|
Number | 35 | 39 | 35 | 34 | 21 |
Percentage | 87.5 | 97.5 | 87.5 | 85 | 52.5 |
As a result of the above analysis, it can be confirmed that the research content of picture semantics is to study the picture, text, sound, image, interactive form, and the factual knowledge, conceptual knowledge expressed, or transmitted in the multimedia picture. The relationship and matching rules between procedural knowledge and metacognitive knowledge and the semantic rules suitable for different knowledge types are summarized. Through the specification of semantic rules, the design of multimedia picture can present and express all kinds of knowledge more accurately as follow. Student achievement statistics is shown in Table
Student achievement statistics.
S1 | S2 | S3 | S4 |
---|---|---|---|
100 | 80 | 90 | 92 |
95 | 85 | 91 | 93 |
93 | 89 | 94 | 96 |
98 | 93 | 98 | 99 |
It can be seen from Table
Performance of the teaching system based on image linguistics.
As can be seen from the Figure
It can be seen from the Figure
The relationship between the image linguistic teaching system and time.
Comparison of the accuracy of the model established in this article and other methods.
We compare the model established in this article with the methods in the latest international references, such as support vector regression machine (SVR) and extreme learning machine (ELM). The comparison uses model-based accuracy as an indicator. Figure
Multimedia picture linguistics is composed of three parts: construction, semantics, and pragmatics, which correspond to the art law of multimedia picture, the cognitive law of multimedia picture, and the humanization and naturalization of picture design. Therefore, the study of this paper is an important part of creating multimedia picture linguistics. At present, the research on the theoretical system of multimedia picture linguistics in China is very limited. In order to further improve and develop the theoretical system of multimedia picture linguistics, this paper draws lessons from the research results of educational technology, linguistics, psychology, and semiotics. Then, starting from the research methods and theoretical framework of multimedia technology and picture linguistics, the relationship between multimedia learning cognitive theory and Mayer's multimedia learning cognitive theory is deeply explored and demonstrated. Experiments have proved that the teaching system that combines picture linguistics and multimedia technology proposed in this paper can provide a framework, ideas, and methods for the research of multimedia picture linguistics and multimedia technology. It is an information teaching application. The method proposed in this paper can provide more frameworks, methods, and ideas to construct a teaching system that combines picture linguistics and multimedia technology and make it a language tool for developing and designing multimedia learning materials. In the future, we will further devote ourselves to related research to provide reference and reference for the development of the education industry.
The data used to support the findings of this study are available from the corresponding author upon request.
The author(s) declare(s) that they have no conflicts of interest.