Press release April 18, 2016
Fixstars Corporation (Head Office: Shinagawa-ku, Tokyo, President: Satoshi Miki, hereinafter Fixstars) and Nautilus Technologies, Inc. (Headquarters: Shinagawa-ku, Tokyo, CEO: Takeshi Kamabayashi, hereinafter Nautilus) have developed the framework "Asakusa on M3BP" for implementing the operation and development of the core batch processing on a parallel distributed processing platform and the execution platform "M3® for Batch Processing (M3 for BP)", and released it as open source software. Asakusa on M3BP achieves cost effectiveness several ten times not only by scale-up solutions but also by scaled-out solutions based on parallel distributed processing due to faster processing and reduced TCO.
Some of the backbone batch processing takes place several hours to several tens of hours on hosts, mainframes, and databases. In Nautilus, we have realized that by combining Asakusa Framework™ (*1) with MapReduce and Spark™ of Hadoop®, these time-consuming mission-critical batch processing can be shortened several tens of levels. However, MapReduce targets TB to PB, Spark targets large data of hundreds of GB to TB, and small scale data (several GB to several tens of GB) is not the target. Therefore, even if Asakusa Framework is applied to mission-critical batch processing accompanied by this small-scale data, the overhead associated with distributed processing becomes relatively large, and there was a limit in speeding up.
Asakusa on M3BP achieves high-speed standard batch processing that handles data of several GB to several tens of GB. Comparisons between batch processing time using actual business data revealed that in the Asakusa Framework, it took several hours in the relational database, about 40 minutes in combination with MapReduce, less than 4 minutes in the case of Spark, and in the case of M3 for BP, it shortened to less than 2 minutes. In addition, since M3 for BP performs parallel processing using multi-core / multiprocessor on a single node, it does not require multiple servers like MapReduce and Spark, which further reduces the cost of constructing and operating the environment.
For comparison of performance results, see separate paper white paper (PDF:456KB) open_in_new (Japanese) refer to the section on internal performance evaluation.
Asakusa Framework realizes portability of programs, applications developed using Asakusa Framework can run the same program on MapReduce, Spark, M3 for BP processing platforms. First of all, starting with Asakusa on M3BP, which is low in implementation cost and cost effective, you do not need to change the application even if you want to move the execution base to Spark or MapReduce as you need more processing .
Asakusa on M3BP is released as OSS. For more detailed information, please refer to the following.
（*1）The Asakusa Framework™ is a framework for large scale batch processing on parallel distributed processing infrastructure and is open source. Asakusa Framework™ is compatible with MapReduce and Spark's parallel distributed processing engine, and now M3 for BP is newly added as a processing engine.
（*2）Hadoop is a software infrastructure for efficiently distributing and managing large-scale data developed and published by the Apache Software Foundation. It is open source software, and anyone can freely acquire and use it.
※Asakusa Framework™is a registered trademark of Nautilus Technologies, Inc.
※Hadoop®、Spark™ are registered trademarks or trademarks of the Apache Software Foundation.
※M3® is a registered trademark of Fixstars Corporation