Fig 5 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Download
View publication
The process of using floating-point 32-bit default parameters vs. using clustered parameters.

The process of using floating-point 32-bit default parameters vs. using clustered parameters.

Source publication
Fig. 1. Application of the transformer architecture to image...
Fig. 2. Execution time breakdown of the DeiT and ViT models.
Figure 3 shows.
Fig. 4. K-means clustering algorithm grouped the data into three...
+4 Fig. 5. The process of using floating-point 32-bit default parameters...
Improving the Efficiency of Transformers for Resource-Constrained Devices
Preprint
Full-text available
  • Jun 2021
Transformers provide promising accuracy and have become popular and used in various domains such as natural language processing and computer vision. However, due to their massive number of model parameters, memory and computation requirements, they are not suitable for resource-constrained low-power devices. Even with high-performance and specializ...
Cite
Download full-text

Contexts in source publication

Context 1
... use the clustered parameters, the corresponding 8-bit index is fetched instead of the 32-bit parameter in the baseline. The index is used to pick the corresponding centroid from the very small table of centroids as shown in Figure 5. In this paper, we explore scalar clustering in which every single parameter in the model will be directly represented by an index. ...
Context 2
... it shows, despite extra instructions and overhead in the kernel to perform the indirect accesses, as shown in Figure 5, the reduced pressure on the memory system, because of clustered parameters, provides significant benefit specially in GPUs with more computing resources such as the GPU in Conf-3. To demonstrate the advantages of using clustered data, the results are obtained while putting maximum pressure on the memory subsystem. ...

Similar publications

View
Figure 1. ReLU activation function.
Figure 2. Illustration of dataset, (A)-(F) are images of chickens at...
Figure 3. Illustration of different dataset augmentation methods. (A)...
Figure 4. Illustration of our methods.
+11 Figure 5. Illustration of inception.
A High-Performance Day-Age Classification and Detection Model for Chick Based on Attention Encoder and Convolutional Neural Network
Article
Full-text available
  • Sep 2022
Thanks to the boom of computer vision techniques and artificial intelligence algorithms, it is more available to achieve artificial rearing for animals in real production scenarios. Improving the accuracy of chicken day-age detection is one of the instances, which is of great importance for chicken rearing. To solve this problem, we proposed an att...
View
View
View
Figure 2
Figure 3
Figure 4
Figure 5
Figures
Tissue Segmentation from Whole-Slide Images Using Lightweight Neural Networks
Preprint
Full-text available
  • Dec 2020
p style='margin:0in;line-height:150%;font-size:15px;font-family:"Arial",sans-serif;margin-left:.5in;background:white;'> Abstract —Pathology slides of malignancies are segmented using lightweight convolutional neural networks (CNNs) that may be deployed on mobile devices. This is made possible by preprocessing candidate images to make CNN analysis t...
View