===========
When using a GPGPU card e.g. Nvidia, is it possible to use the GPU on the attached card across multiple processes at the same time ? In other words, say I have two applications each requiring some number of GPU cores. Can these two applications use the GPU simultaneously ? This question has further two parts:
(a) can the same GPU core be shared across multiple processes ?
(b) can a different set of cores on a GPU be shared across multiple processes ?
I am comparing against the GPP processor where multiple processes are multiplexed on the same core by the OS (context-switched).
(a) can the same GPU core be shared across multiple processes ?
(b) can a different set of cores on a GPU be shared across multiple processes ?
I am comparing against the GPP processor where multiple processes are multiplexed on the same core by the OS (context-switched).
----------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------
Answer:
Below are two excerpts from Fermi White Paper ! I have highlighted some keywords !
10x Faster Application Context Switching
Like CPUs, GPUs support multitasking through the use of context switching, where each
program receives a time slice of the processor’s resources. The Fermi pipeline is optimized to
reduce the cost of an application context switch to below 25 microseconds, a significant
improvement over last generation GPUs. Besides improved performance, this allows
developers to create applications that take greater advantage of frequent kernel-to-kernel
communication, such as fine-grained interoperation between graphics and PhysX applications.
Concurrent Kernel Execution
Fermi supports concurrent kernel execution, where different kernels of the same application
context can execute on the GPU at the same time. Concurrent kernel execution allows
programs that execute a number of small kernels to utilize the whole GPU. For example, a
PhysX program may invoke a fluids solver and a rigid body solver which, if executed
sequentially, would use only half of the available thread processors. On the Fermi architecture,
different kernels of the same CUDA context can execute concurrently, allowing maximum
utilization of GPU resources. Kernels from different application contexts can still run
sequentially with great efficiency thanks to the improved context switching performance.
I am trying to address your questions below: My comments are in blue:
When using a GPGPU card e.g.Nvidia, is it possible to use the GPU on the attached card across multiple processes at the same time ? In other words, say I have two applications each requiring some number of GPU cores. Can these two applications use the GPU simultaneously ?
Different kernels of the same CUDA application context can execute on the GPU at the same time.
Kernels from different application contexts run sequentially but with context switching.
This question has further two parts:
(a) can the same GPU core be shared across multiple processes ?
One GPU core can not be shared across multiple processes.I can elaborate the complete execution phase if required !
(b) can a different set of cores on a GPU be shared across multiple processes ?
A minimal set of GPU cores in Fermi, on which a kernel can be launched , comprises of 32 cores on an SM.There are 16 SMs and hence a maximum of 16 different concurrent kernels can be launched.
The answer to your question depends on whether the multiple processes(kernels) are from same application or different application.
If the multiple processes(kernels) are from same application, the processes are divided across set of cores (32 cores on one SM). ie set of 32 cores on a GPU are shared across multiple processes.
If the multiple processes(kernels) are from different application, they can not be divided concurrently among cores.
==============
The following two implementations are worth considering if the resources are available.
a) If one has single application, Divide different kernels of same application among multiple GPU cards
b) If one has multiple applications, Launch different applications on different GPU cards
No comments:
Post a Comment