How max-num-seqs control GPU mem useage? Why increase max-num-seqs will use less memory? With my test: max-num-seqsGPU mem(Gib) 25620.6 204819 409613 TaChaoadded theusageHow to use vllmlabelMar 19, 2024 Collaborator hmellorcommentedApr 20, 2024 ...
Well theif seq_idx >= self._num_seqs:can simply be changed to how we usually do it, viais_less_than_num_seqs. Actually I just checked the code, and we already do that, where we handle theget_data_seq? So we can just remove this check. 👍1 Member albertzcommentedSep 5, 2024 ...
在当前目录下新建两个文件夹 -max_target_seqs、num_alignments,为了比较这两个参数的不同,我们分别把-max_target_seqs和-num_alignments从1到500,分别运行500次,比较这500个结果之间的异同。 2.3.1-max_target_seqs $ for i in `seq 1 500`; do blastn -query otu.fa -outfmt '6 qseqid sseqid qst...
另一个问题：在我们的测试中，推测性解码比纯原模型慢。