Parallel architecture 

Context in source publication

Context 1

... supporting random access, number encoding in native format, data compression, individual data set encryption, and storage strategies for parallel I/O and multidimensional data structures; the current version is HDF5. There is two important feature of HDF. First is that files can contain binary data as multi-dimensional arrays and allow direct access to parts of the file without first parsing the entire contents [1]. Second feature is supporting for standard parallel I/O interfaces. Known as The Parallel Hierarchical Data Format 5 (Parallel HDF5), for work with it is required MPI/IO interface, which is supported through MPICH ROMIO [9]. Now days, ROMIO has support to most common file system, but only offer bracing to PVFS as distributed file system. The purpose of Parallel HDF5 is to make it easy for users to use the library, providing compatibility with serial HDF5 file. One approach is to read and write data by hyperslab [1], i.e., a multidimensional array that can be spread by rows, columns, patterns and chunks, and a hyperslab selection could be a logically contiguous collection of points, or it can be a regular pattern of points or blocks, depending on the type used. When works with parallel I/O, the properties of communication that will realize the I/O operation its important to synchronise the nodes. The Parallel HDF5 library has available two types of properties (collective and independent data access) and four hyperslab model (Contiguous Hyperslab, Regularly Spaced Data, Pattern and Chunk) [1]. In HDF5 there are two essential structures which form the base for the library: dataset and group. Dataset is a multi-dimensional array of datatype; HDF stores and organize all kinds of data from atomic to composed types, similar to the C structure construct. Special array operations, such as chunks, compression and extendibility are available through the HDF library and can be applied to a dataset. Other important structure is the dataspace. Through a dataspace is possible to require components of dataset or even an attributes are defined, as well as array ranks, sizes and types. The group is similar to UNIX directories, though cycles are allowed. Every file is started with a root group, represented as /, and could be followed by the name of another group or a dataset as show in figure 2. The works related are suchlike to our work, concerning with the use of parallel I/O as a solution for I/O bottlenecks access for large amounts of stored data. The first related work presented in Nikhil Laghave [10], is very similar to our work. This work is focused on the use of a parallel I/O library for scalability issues involving fermion dynamics for nuclear structure (MFDn). This work used the HDF5 parallel version for parallel I/O, testing with collective and independent models. As result once file sizes increasing above 20 GB, parallel HDF5 becomes more cost-eective than sequential I/O for sufficiently large datasets than sequential binary I/O. The work of A. Adelmann [11] focused on using parallel I/O for particle-based accelerator simulation which involved vast quantities of data and dimensional arrays. He used parallel I/O performance for MPI code as well parallel HD5 by developed API call H5part. H5part is a portable high performance parallel data interface for particle simulation. He compared read and write performance in simulations between H5part, mpi-io and one file per process. HDF5 showed good performance in writing, though mpi-io showed better results. Finally, H. Yu [12] even not using parallel HDF as solution, he demonstrated an interesting work which dealt with large earthquake simulations. He faced scalability issue and I/o bottleneck, due earthquake simulation required large file to storage. To solve his problem, he developed an application using parallel I/O strategies through MPI I/O to address his needs. The results was considerable to remove the I/O bottleneck and also hide pre-processing costs. Our work comes to propose a new architecture for Macedos approach, in order to avoid I/O bottlenecks and get better performance using parallel data access to HDF files stored in the PVFS distributed file system. To accomplish our work, we configured MPI environment to work with Parallel HDF5. As cited in session 2.3, the Parallel HDF5 library requires a parallel MPI/IO interface through ROMIO and when working with MPI, it is necessary to design it to running in a cluster environment. It is important to note the requirement to use the mpirun shell script to run any MPI application, which attempts to hide the differences in starting jobs for various devices from the user [13]. We created an additional procedure to work with the CyclopsDCMServer. This procedure should be called every time when is required to retrieve or store some medical information. For now, we build an application concerning with I/O access, which our application just responsible to direct access a file for reading and writing a dataset. Figure 4 illustrates how the architecture works. It was crucial to modify the H5WL functionality. Instead of having the H5WL responsible for reading and writing the binary information created by PACS, it will be treated as a new parallel application. When some client requests to store a DICOM files, it necessary to initiate the parallel application by calling mpirun shell script. After, all communication between them will be made by socket connections. The communication is done by the master process (represented by MPI process zero) and H5WL, which the messages is represent function parameters, like the location that is the target of an operation (group path), the image buffer and the number of MPI processes. In write functions, per example, the H5WL will first receive the DICOM file, create a new hierarchy of the image based on DICOM file layers, get the path location for new image (JPEG image) and then call mpirun procedure to start the MPI application. The master process will delegate the communication with H5WL to retrieve the function to perform, get the location (group) of image in the HDF5 structure, the arguments for the job and the stream of images to be stored. Receiving the stream, the master node has to distribute the memory buffer to each process in small buffer equal to number of nodes. Finally, once the jobs have executed, the main process will return to the wrapper the status of reading or writing the buffer. Our experiments are based on the Parallel HDF5 architecture, adapted to use PVFS, MPI and sequential CyclopsDCMServer. The parallel environment consists in four node cluster, as specified in Table 1 and one dedicated computer for DICOM server. Unfortunately, the environment is non-dedicated cluster and belongs to Telemedicine Laboratory and share a connection network 100 Mbs Ethernet. The operating system installed on all nodes is CentOS with kernel 2.6.32, and there only one metadata PVFS node. Each node has a PVFS client for access to the PVFS file system and MPI compiled. It is important to note that our results do not take into consideration external factors, like computer users, using the same ...