In recent years, expectations have increased regarding the analytical processing of big data, which provides added value by processing and analysing large quantities of time-series data on a real-time basis. This type of processing has been launched in areas that include the automated trading (algorithmic trading) of securities companies’ stock and the analyses of traffic by network providers. The utilisation of this processing is also being considered in fields such as healthcare and public security. These industries require high-speed analytical processing for accurate information analysis, even as data volume increases. This processing must also flexibly respond to rapidly changing analysis requests.
The use of dedicated hardware is expected to accelerate the processing of big data by 10 to 50 times compared with processing using software. However, the development and implementation of new hardware has historically taken a significant amount of time, often requiring several months. Insufficient flexibility has also plagued hardware as it did not permit changes to content during processing.
"NEC’s newly developed technologies permit the easy design of hardware dedicated to high-speed processing. In order to carry out data analysis, users simply input the required analytical processing content using SQL, a programming language used as an open interface," said Naoki Nishi, General Manager, Green Platform Research Laboratories, NEC Corporation. "This reduces the time for developing hardware from as much as several months to approximately 1/50 the time, or several hours. The technology also reduces the time required for rewriting the processing content to approximately 1/1,000,000 of the previous time, or from milliseconds to nanoseconds, allowing the dynamic operation of the system without shutting it down."
Main features of this NEC technology are as follows:
1) Permits the use of SQL for designing hardware, reducing design time to approximately 1/50 of the previous time
NEC has developed design technology that permits the use of CyberWorkBench, a circuit synthesis technology for the Field Programmable Gate Array (FPGA) owned by NEC, to automatically convert software designed using SQL, which is widely used for the analytical processing of big data, to dedicated hardware on FPGA. This permits users in charge of data analysis to design FPGA directly using SQL, a familiar programming language. This means that circuit design work by engineers is no longer necessary, and the time required for developing hardware, which conventionally takes several months, is reduced to approximately 1/50 of the time, or several hours.
2) Processing content can be changed dynamically without shutting down the system
NEC has developed a hardware processing mechanism that switches between data processing content instantaneously, while operating the circuit for the old processing and a circuit for the new processing in parallel. Previously, processing had to be halted for several milliseconds when changing processing content since the circuit for the old processing had to be reconfigured for the new processing. The newly developed mechanism reduces the shutdown time to nanoseconds, or approximately 1/1,000,000 of the previous time. This enables dynamic corrections and processing changes without shutting down the system.
NEC will continue to carry out research and development with the aim of providing the technology as a real-time big data processing solution using hardware by the close of the fiscal year ending March 2015.