Figure 4 - uploaded by Albert Wegener
Content may be subject to copyright.
APAX Encoder Block Diagram
Source publication
In the multicore era, the time to computational results is increasingly
determined by how quickly operands are accessed by cores, rather than by
the speed of computation per operand. From high-performance computing
(HPC) to mobile application processors, low multicore utilization rates
result from the slowness of accessing off-chip operands, i.e. t...
Citations
... Compared with SZ and ZFP, DCT-EC obtains a higher range between 110 and 120 on most datasets. From Table III, we can see that all three compressors have "five nines" [42] or better correlations with all three P s. For DCT-QT, it obtains PSNRs of no more than 50 and Pearson correlations of no more than 0.9999 on six datasets (with max θ of around 1E−3), which shows its limitation in compression precision compared with DCT-EC. ...
... This coefficient is a measurement of the linear dependence between two variables, giving ρ between +1 and −1, where ρ = 1 is the total positive linear correlation. The APAX profiler [Wegener (2013)] suggests that the correlation coefficient between original and reconstructed data should be 0.99999 ("five nines") or better. ...
... Table 5 shows the Pearson correlation coefficients evaluated by Z-checker for different lossy compressors with different maximum compression errors on the CESM and Hurricane data sets. The APAX profiler [Wegener (2013)] suggests that the correlation coefficient between original and reconstructed data should be 0.99999 ("five nines") or better. Thus, to satisfy this suggestion, we should set the value-range-based error bound to be around 10 −4 or lower for the SZ and ZFP compressors. ...
Because of vast volume of data being produced by today's scientific simulations and experiments, lossy data compressor allowing user-controlled loss of accuracy during the compression is a relevant solution for significantly reducing the data size. However, lossy compressor developers and users are missing a tool to explore the features of scientific datasets and understand the data alteration after compression in a systematic and reliable way. To address this gap, we have designed and implemented a generic framework called Z-checker. On the one hand, Z-checker combines a battery of data analysis components for data compression. On the other hand, Z-checker is implemented as an open-source community tool to which users and developers can contribute and add new analysis components based on their additional analysis demands. In this paper, we present a survey of existing lossy compressors. Then we describe the design framework of Z-checker, in which we integrated evaluation metrics proposed in prior work as well as other analysis tools. Specifically, for lossy compressor developers, Z-checker can be used to characterize critical properties of any dataset to improve compression strategies. For lossy compression users, Z-checker can detect the compression quality, provide various global distortion analysis comparing the original data with the decompressed data and statistical analysis of the compression error. Z-checker can perform the analysis with either coarse granularity or fine granularity, such that the users and developers can select the best-fit, adaptive compressors for different parts of the dataset. Z-checker features a visualization interface displaying all analysis results in addition to some basic views of the datasets such as time series. To the best of our knowledge, Z-checker is the first tool designed to assess lossy compression comprehensively for scientific datasets.
... This coefficient is a measurement of the linear dependence between two variables, giving ρ between +1 and −1, where ρ = 1 is the total positive linear correlation. The APAX profiler [17] suggests that the correlation coefficient between original and reconstructed data should be 0.99999 ("five nines") or better. Metric 4: To evaluate the size reduce as a result of the compression, we use the compression factor CF , ...
... In this paper we study two compression algorithms designed for floating-point data: Samplify's APAX (APplication AXceleration) encoder [30, 31] and the fpzip [19, 20] compressor developed at LLNL. The APAX algorithm has previously been applied to climate data [10] , computed tomography x-ray samples [33], and a variety of integer and floating-point datasets [32] . We describe these two compressors in parallel to highlight their similarities and differences. ...
... " Analogous to how it is possible to analyze the impact mesh resolution has on accuracy, we require models to predict how quantization and variable-precision arithmetic influences the simulation state and error growth over time. The APAX Profiler [32] has been used successfully to predict acceptable compression rates for other media by analyzing the intrinsic noise level relative to the signal strength. However , such tools do not account for the potential cascading effects of compression-induced errors that grow over time, nor how these errors correlate with physics-based metrics. ...
This paper examines whether lossy compression can be used effectively in physics simulations as a possible strategy to combat the expected data-movement bottleneck in future high performance computing architectures. We show that, for the codes and simulations we tested, compression levels of 3-5X can be applied without causing significant changes to important physical quantities.
Rather than applying signal processing error metrics, we utilize physics-based metrics appropriate for each code to assess the impact of compression. We evaluate three different simulation codes: a Lagrangian
shock-hydrodynamics code, an Eulerian higher-order hydrodynamics turbulence modeling code, and an Eulerian coupled laser-plasma interaction code. We compress relevant quantities after each time-step to approximate the effects of tightly coupled compression and study the compression rates to estimate memory and disk-bandwidth reduction. We find that the error characteristics of compression algorithms must be carefully considered in the context of the underlying physics being modeled.
