Carp

Decoupling Architecture

  • Motivation
    • Generalizing "all-to-all computation"
      • Lowering the threshold to use the K computer
  • Decoupling Architecture
    • Decoupling the kernel code and the other system programing staff

Carp framework

  • pCarp
    • Parallel version for production run
  • sCarp
    • Sequential version for kernel development and debugging

Code Skeleton

  • Users do not care about
    • Programming Language
    • MPI
    • Load Balance
    • I/O

pCarp Performance

  • Original Program
    • 240 Sec.
  • pCarp + Sequential Programs
    • 430 Sec. (+190 Sec.)
      • Pipe Transfer: 159 GBytes
      • Pipe BW: 1 GB/s
      • Most of the overhead comes from the transfer via the pipe (160 Sec.) !!

Pipe Performance


Summary

  • Decoupling Architecture
    • Users do not to worry about the parallel programing (I/O, communication, loadbalancing, ...)
    • Users can write their programs in any programing languages

Future Work

  • [ps]Carp improvement - in the future
    • Increasing the pipe buffer size in the Linux kernel
      • fcntl( fd, F_SETPIPE_SZ )
        • Linux 2.6.35~, or
    • Developing new "popen" implementation to improve the bandwidth
  • Having applications other than XFEL