Figure 4 - uploaded by Hariri Walid
Content may be subject to copyright.
Transformer architecture.

Transformer architecture.

Source publication
Preprint
Full-text available
Large language models have revolutionized the field of artificial intelligence and have been used in various applications. Among these models, ChatGPT (Chat Generative Pre-trained Transformer) has been developed by OpenAI, it stands out as a powerful tool that has been widely adopted. ChatGPT has been successfully applied in numerous areas, includi...

Contexts in source publication

Context 1
... advantage of the Transformer is its parallelizability, which allows it to handle large-scale pre-training and adaptability to different downstream tasks without inductive biases. Figure 4 presents the architecture of the transformer. The Transformer architecture has become the primary choice in natural language processing due to its ability to learn and parallelize. ...
Context 2
... generate a prompt, the user should give an indication, and the generator will provide a convenient prompt for ChatGPT. Figure 14 presents an example of prompt generation using Hugging face. ...

Similar publications

Preprint
Full-text available
Deep Learning (DL) , a variant of the neural network algorithms originally proposed in the 1980s, has made surprising progress in Artificial Intelligence (AI), ranging from language translation, protein folding, autonomous cars, and more recently human-like language models (CHATbots), all that seemed intractable until very recently. Despite the gro...

Citations

... These are general categories of generative AI based in input prompts. Walid (2023) explains that "unimodal models take prompts from the same modality as the content they generate, while multi-modal models can accept prompts from different modalities and produce results in multiple modalities" as shown in Figure 5.MODAL. OpenAI's ChatGPT is an example of a Transformer-based, neural network application that uses a multimodal generative process. ...
... MODALUnimodal vs. Multi-Modal Generative AI ModelsNote. Source:Walid, 2023 architecture Generative AI natural language processing (NLP) Large Language Model (LLM) ...
Preprint
This document will be used by MARSOC and the SOF enterprise to develop an enterprise baseline of knowledge and understanding about Autonomy and AI. The purpose of the release is to implement the AIRI scale prior to the next round of small unmanned systems (SUMS) requirements set to go into staffing in March of 2024. This document will be briefed and released to the SOF enterprise on or about 5 Feb 2024.
Article
Full-text available
This paper investigated how to design undergraduate assessments that could not be adequately answered by ChatGPT. The paper was embedded in the revised Bloom’s Taxonomy as a theoretical model. ChatGPT has presented significant challenges to lecturers when setting assessments at the tertiary level. There is enormous potential for students to attempt to use ChatGPT to write and pass assessments. Design Based Research formed the basis of this paper’s research design. Qualitative research methods were used to interpret recordings of interaction with ChatGPT during Hermeneutic research cycles. The paper found that it was possible to design an assessment that could not be satisfactorily answered solely by ChatGPT. Interactions with ChatGPT were found to be an essential aspect of the research process. In light of the findings, efforts should be made to revise undergraduate assessments to encourage students to engage with them rather than copy and paste from ChatGPT. The paper recommended ChatGPT as a useful tool or form of interactive Google that can support writing assessment but is unable to replace the student effectively. The paper suggests that students should receive training on the limitations of ChatGPT and how they can use it more effectively in their studies.