Open Stack

On Wed, 2021-09-01 at 20:37 +0800, 陈克 wrote:
> Hi, 
> I recommend a tool to test GPU performance, which depends on CUDA installation.
> 
> 
> link: https://github.com/GPUburn/gpuburn/tree/master/GPUBurn
> 
> 
> 
if youtube hard ware review have show us anything the best way to benchmark gpu hardware is actully
to use repsentivte application rather then synteic benchmarks as gpu vendors have a habbit of tuninging
the gpu driver to perfrom better in popular syntitich benchmarks.

i would suggest selecting a representive set that cover a number of usecase from cad, gaming, to video transcoding to ml (inference and  trianing) to compute vission.

blender is a popular opensouce 3d modelely software and it provides some in build bench marks that can be used for evaulating the performace of 3d cad and rendering.
opencv is a popular toolkit for compute vission and they have a numbner of benchmark examples such as https://docs.opencv.org/4.5.2/dc/d69/tutorial_dnn_superres_benchmark.html
on the more pure ml side there are some syntetic benachmarks provided by tensorflow https://github.com/tensorflow/benchmarks/tree/master/perfzero
but often a better approch is to use a common opensocue data set and model and messue the time it takes to  train the same model with the same data set in differnet environemnts
you also want to messure the infernce rate which involved takign a pre trained model and feeding it data form out side of its training set and mesurrign the performance.
its important that that model and data used for the inferce is the same used across all your deployments.
pytouch is also another popular ai/ml framework that can be used in a simialr way.

looking quickly at https://github.com/GPUburn/gpuburn/tree/master/GPUBurn it looks liek its actully not a benchmark utility but instead a stress test tool for mesuring stablity which is a very
differnt thing and i dont think it would be helpful in your cases. in fact using it in a public cloud for example could be considered a break of fair use since its really desiing to
put the gpu under as much stress as possibel to detect fault hardware which will impact other users of the gpu/

most of the gaming/windows based tool are propritaty which makes them hard to use as benhcmarks in an opensouce project due to licensing so they likely shoudl be avoided.

but ya my advice would be dont look for benchmarks in general but look for framworks/tools/applcation that peopel use to do actual useful work that can be sued
to executate a repatable action like useing handbrake to transcode a video file form one format to another to messur the hardwarre encoder performance with fixed inputs
then use that out in this case transcoding time as one aspect of the performance mesurment to establish your benchmark.


> 
> 
> 
> 
> 
> Thanks,
> Ke Chen
> 
> 
> 
> 
> 
> 
> 
> 在 2021-08-31 23:10:03，"Sylvain Bauza" <sbauza at redhat.com> 写道：
> 
> 
> 
> 
> 
> On Tue, Aug 31, 2021 at 4:34 PM Ildiko Vancsa <ildiko.vancsa at gmail.com> wrote:
> 
> Hi,
> 
> As we are approaching the end of the holiday season I wanted to surface back my question about GPU performance testing. Does anyone have any hints to find the best tools to do some benchmarking with?
> 
> 
> 
> 
> 
> 
> You made the point, Ildiko, I was on a long-running time-off so I didn't had time to look at your question yet.
> 
> 
> Good concern tho, I have no knowledge about this, but I can ping a few other folks to get you an answer.
> -Sylvain
> 
> 
> Thanks,
> Ildikó
> 
> 
> > On Aug 9, 2021, at 08:46, Ildiko Vancsa <ildiko.vancsa at gmail.com> wrote:
> > 
> > Hi,
> > 
> > I got a question about tools and practices to check GPU performance in an OpenStack environment that I need some help to answer.
> > 
> > The question is about recommended GPU performance testing/benchmarking tools if there are a few that people in the community are using and would recommend? The scope of the testing work is to check GPU performance in OpenStack VMs (both virtualized and passthrough).
> > 
> > All the help and pointers are very much appreciated!
> > 
> > Thanks,
> > Ildikó
> > 
> > 
> 
>