getInput的参数0代表输入的张量索引,因为我们的输入输出都只有一个张量,因此填索引0即可。最后创建TensorRT最关键的用于推理的Engine(引擎) auto engine = builder->buildEngineWithConfig(*network, *config); 至此,构建阶段就算完成了。至于engine的序列化为模型文件以及通过读取模型文件来
int32_t inputIndex = engine->getBindingIndex(INPUT_NAME); int32_t outputIndex = engine->getBindingIndex(OUTPUT_NAME); 使用这些索引,设置一个缓冲区数组,指向 GPU 上的输入和输出缓冲区: void* buffers[2]; buffers[inputIndex] = inputBuffer; buffers[outputIndex] = outputBuffer; 然后,您可以调用 ...
// input and output tensors. //获取输入,输出tensor索引 int inputIndex = engine->getBindingIndex(INPUT_LAYER_NAME), int outputIndex = engine->getBindingIndex(OUTPUT_LAYER_NAME); //申请GPU显存 // Allocate GPU memory for Input / Output data void* buffers = malloc(engine->getNbBindings() *...
//获取输入,输出tensor索引 int inputIndex = engine->getBindingIndex(INPUT_LAYER_NAME), int outputIndex = engine->getBindingIndex(OUTPUT_LAYER_NAME); //申请GPU显存 // Allocate GPU memory for Input / Output data void* buffers = malloc(engine->getNbBindings() * sizeof(void*)); cudaMalloc(...
getNbBindings() == 2); void* buffers[2]; // In order to bind the buffers, we need to know the names of the input and output tensors. // Note that indices are guaranteed to be less than IEngine::getNbBindings() const int inputIndex = engine.getBindingIndex(INPUT_BLOB_NAME); ...
intoutputIndex = engine->getBindingIndex(OUTPUT_LAYER_NAME); //申请GPU显存 // Allocate GPU memory for Input / Output data void* buffers =malloc(engine->getNbBindings() *sizeof(void*)); cudaMalloc(&buffers[inputIndex], batchSize * size_of_single_input); ...
int output_index = engine_infer->getBindingIndex("output0"); //1 //engine模型动态batch(BATCH_SIZE, 3, width, height) nvinfer1::Dims inputSize=engine_infer->getBindingDimensions(input_index); nvinfer1::Dims outputSize = engine_infer->getBindingDimensions(output_index); ...
CUDA_CHECK(cudaMemcpyAsync( (void*)(matClass.data), mCudaBuffers[2], mBindBufferSizes[2], cudaMemcpyDeviceToHost, mCudaStream)); 1. 2. 1. 通过Engine mEngine->getBindingIndex(tensorName.c_str())通过输出tensor的字符串获得output tensor 的 index...
输出tensor索引int inputIndex=engine->getBindingIndex(INPUT_LAYER_NAME),int outputIndex=engine->getBindingIndex(OUTPUT_LAYER_NAME);//申请GPU显存// Allocate GPU memory for Input / Output datavoid*buffers=malloc(engine->getNbBindings()*sizeof(void*));cudaMalloc(&buffers[inputIndex],batchSize*size...
解决方法:对于动态形状的输入,需要在推理前设置具体的形状。可以使用setBindingDimensions函数来设置动态输入的维度。例如: cpp engine->getContext()->setBindingDimensions(inputIndex, inputDims); 通过遵循以上指导和示例代码,你应该能够正确地使用getBindingDimensions函数来获取TensorRT中绑定的维度信息。