“FLOPS to BYTES: Accelerating Beyond Moore’s Law”


Satoshi Matsuoka
Professor, Tokyo Institute of Technology and Fellow, Advanced Institute for Science and Technology, Japan

Abstract – The so-called “Moore’s Law”, by which the performance of the processors will increase exponentially by factor of 4 every 3 years or so, is slated to be ending in 10-15 year timeframe due to the lithography of VLSIs reaching its limits around that time, and combined with other physical factors. This is largely due to the transistor power becoming largely constant, and as a result, means to sustain continuous performance increase must be sought otherwise than increasing the clock rate or the number of floating point units in the chips, i.e., increase in the FLOPS. The promising new parameter in place of the transistor count is the perceived increase in the capacity and bandwidth of storage, driven by device, architectural, as well as packaging innovations: DRAM-alternative Non-Volatile Memory (NVM) devices, 3-D memory and logic stacking evolving from VIAs to direct silicone stacking, as well as next-generation terabit optics and networks. The overall effect of this is that, the trend to increase the computational intensity as advocated today will no longer result in performance increase, but rather, exploiting the memory and bandwidth capacities will instead be the right methodology. However, such shift in compute-vs-data tradeoffs would not exactly be return to the old vector days, since other physical factors such as latency will not change when spatial communication is involved in X-Y directions. Such conversion of performance metrics from FLOPS to BYTES could lead to disruptive alterations on how the computing system, both hardware and software, would be evolving towards the future.


“Japanese plans for Open HPC and Big Data / AI Infrastructure”

Short Talk.

Satoshi Matsuoka
Professor, Tokyo Institute of Technology and Fellow, Advanced Institute for Science and Technology, Japan


Abstract – Japanese investment into public, open science HPC infrastructures for research and academia have had a long history, in fact longer than that of the United States. The current nation-wide infrastructure, namely HPCI (High Performance Computing Infrastructure of Japan) revolves around a consortium similar to XSEDE in the US, including the Tier 0 K-Computer and the Tier 1 HPC centers located at 9 major universities as members, all of which are interconnected with 100Gbps SINET5 national academic network as well as a 20+ Petabyte nationwide shared storage.
There is a continued plan for long-term investment into the infrastructure, including the Post-K supercomputer planned to be operational around 2021, as well as top-tier  university machines such as Univ. Tokyo’s Oakforest-PACS supercomputer which was just commissioned at 25 Petaflops Peak, and our TSUBAME3 supercomputing at Tokyo Tech who will sport a similar performance in combination with its predecessor, TSUBAME2, both of which I lead the design and development. There now is also a focus on Big Data / AI, with three national-level AI centers by the three Ministries at their national labs, namely AIST’s AIRC (AI Research Center), Riken AIP (AI Project), and NICT’s brain inspired AI research center. In particular, at AIST-AIRC I lead a project of facilitating one of the world’s largest BD/AI focused open and public computing infrastructure called ABCI (AI-Based Bridging Infrastructure). The performance of the machine is slated to be well above 130 Petaflops for machine learning, as well as acceleration in I/O and other properties desirable for accelerating BD/AI.


Satoshi Matsuoka has been a Full Professor at the Global Scientific Information and Computing Center (GSIC), a Japanese national supercomputing center hosted by the Tokyo Institute of Technology, and since 2016 a Fellow at the AI Research Center (AIRC), AIST, the largest national lab in Japan. He received his Ph. D. from the University of Tokyo in 1993. He is the leader of the TSUBAME series of supercomputers, including TSUBAME2.0 which was the first supercomputer in Japan to exceed Petaflop performance and became the 4th fastest in the world on the Top500 in Nov. 2010, as well as the recent TSUBAME-KFC becoming #1 in the world for power efficiency for both the Green 500 and Green Graph 500 lists in Nov. 2013. He is also currently leading several major supercomputing research projects, such as the MEXT Green Supercomputing, JSPS Billion-Scale Supercomputer Resilience, as well as the JST-CREST Extreme Big Data. He has written over 500 articles according to Google Scholar, and chaired numerous ACM/IEEE conferences, most recently the overall Technical Program Chair at the ACM/IEEE Supercomputing Conference (SC13) in 2013. He is a fellow of the ACM and European ISC, and has won many awards, including the JSPS Prize from the Japan Society for Promotion of Science in 2006, awarded by his Highness Prince Akishino, the ACM Gordon Bell Prize in 2011, the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology in 2012, and recently the 2014 IEEE-CS Sidney
Fernbach Memorial Award, the highest prestige in the field of HPC.


Prof. Satoshi Matsuoka