Data Transfer System and Method

High-Speed Data Transfer System Utilizing Thread Scheduling and Memory Buffer Placement per CPU Socket

This technology is a data transfer system that configures both the sender and receiver with multiple CPU sockets and socket-specific memory buffers, assigning master, communication, and I/O threads to each CPU socket to facilitate data transfer between nodes.

When transferring large volumes of data between nodes, bottlenecks often occur between storage, CPU, and memory, leading to reduced transfer efficiency due to memory access latency and memory controller load.

By assigning threads to each CPU socket based on memory buffer location and placing the master and communication threads within the same NUMA node, this technology reduces memory access latency and controller load, thereby increasing transfer efficiency in large-scale data environments such as high-performance computing and big data processing.

‍

‍

Key Features:

• A data transfer system comprising multiple CPU sockets and memory buffers allocated to each CPU socket.
• Each CPU socket includes CPU cores assigned to master threads, communication threads, and I/O threads.
• Each CPU socket is configured to include multiple NUMA nodes composed of the CPU cores within that socket.
• The master thread and communication thread are configured to be assigned to CPU cores within the same NUMA node.

‍

This invention was developed with support from the Ministry of Science and ICT's Core Technology Development Project for Big Data Processing Advancement.

‍