Thursday, 14 April 2011

CUDA Support for Visual Studio 2010


First step for this process is to install Microsoft visual studio 2008. This is to provide
x64 support which is missing in visual studio 2010 and activate this version.
• Download parallel nsight host 1.5 from NVidia website and install it.
• Download parallel nsight monitor 1.5 from NVidia website and install it.
• Now install Microsoft visual studio 2010 using web installer. This process might
take two to three hours. After completing installation activate this version. While
installing make sure to check x64 support.
• In order to create a cuda project first go to file in visual studio menu and then select
a new project then select an empty project and give it a name.
• Now right click on the source file in solution explorer and add a new item. Now
select .cpp file and give it a name.
• Now right click on selection and go to configuration manager. In active solution
platform select new and in following window select x64, copy settings from window
and make sure to check and create new platform.
• Now right click on project and select build customisations and select cuda 3.2.
• Rename .cpp file to .cu and then right click, go to properties and select cuda c/c++.
• Go to project properties, then go to platform tool set and set it to v90.
• Go to project properties select linker, then select linker, then go to input. Now drop
additional dependencies. Without changing anything go to home and type cudart.lib.
• Now go to project properties, select c/c++ and drop target machine and select 64 bit
machine.

CUDA:Sharing a GPU core B/w two processes


===========
When using a GPGPU card e.g. Nvidia, is it possible to use the GPU on the attached card across multiple processes at the same time ? In other words, say I have two applications each requiring some number of GPU cores. Can these two applications use the GPU simultaneously ? This question has further two parts:
(a) can the same GPU core be shared across multiple processes ?
(b) can a different set of cores on a GPU be shared across multiple processes ?

I am comparing against the GPP processor where multiple processes are multiplexed on the same core by the OS (context-switched).

----------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------

Answer:


Below are two excerpts from Fermi White Paper ! I have highlighted some keywords ! 

10x Faster Application Context Switching 
Like CPUs, GPUs support multitasking through the use of context switching, where each 
program receives a time slice of the processor’s resources. The Fermi pipeline is optimized to 
reduce the cost of an application context switch to below 25 microseconds, a significant 
improvement over last generation GPUs. Besides improved performance, this allows 
developers to create applications that take greater advantage of frequent kernel-to-kernel 
communication, such as fine-grained interoperation between graphics and PhysX applications.

Concurrent Kernel Execution 
Fermi supports concurrent kernel execution, where different kernels of the same application 
context can execute on the GPU at the same time. Concurrent kernel execution allows 
programs that execute a number of small kernels to utilize the whole GPU. For example, a 
PhysX program may invoke a fluids solver and a rigid body solver which, if executed 
sequentially, would use only half of the available thread processors.  On the Fermi architecture, 
different kernels of the same CUDA context can execute concurrently, allowing maximum 
utilization of GPU resources. Kernels from different application contexts can still run 
sequentially with great efficiency thanks to the improved context switching performance.


This feature is supported only on GF100 architecture and not on GT200 architecture.

I am trying to address your questions below: My comments are in blue:

When using a GPGPU card e.g.Nvidia, is it possible to use the GPU on the attached card across multiple processes at the same time ? In other words, say I have two applications each requiring some number of GPU cores. Can these two applications use the GPU simultaneously ?

Different kernels of the same CUDA application context can execute on the GPU at the same time.
Kernels from different application contexts run sequentially but with context switching.

This question has further two parts:
(a) can the same GPU core be shared across multiple processes ?

One GPU core can not be shared across multiple processes.I can elaborate the complete execution phase if required !

(b) can a different set of cores on a GPU be shared across multiple processes ?

A minimal set of GPU cores in Fermi, on which a kernel can be launched , comprises of 32 cores on an SM.There are 16 SMs and hence a maximum of 16 different concurrent kernels can be launched.

The answer to your question depends on whether the multiple processes(kernels)  are from same application or different application.

If the multiple processes(kernels) are from same application, the processes are divided across set of cores (32 cores on one SM). ie set of 32 cores on a GPU are shared across multiple processes.

If the multiple processes(kernels) are from different application, they can not be divided concurrently among cores.


==============
The following two implementations are worth considering if the resources are available.
a) If one has single application, Divide different kernels of same application among multiple GPU cards
b) If one has multiple applications, Launch different applications on different GPU cards