免疫检查点阻滞（ICB）通过诱导持久的临床反应，彻底改变了癌症治疗，但仅有少数患者能从ICB治疗中受益。许多研究已确定了各种生物标志物以预测ICB的反应。然而，不同的生物标志物在实践中表现出不同的性能，由于与ICB相关的研究和试验的复杂性，还没有进行及时和公正的评估。在本研究中，我们手动整理了29个已发表的数据集，其中包括来自超过1400名患者的匹配转录组和临床数据，并对这些数据集进行统一预处理以进行进一步的分析。此外，我们收集了39个转录组标志物集，并根据相应的计算方法的性质将其分为基因集类组（分别具有自包含设计和竞争式设计）和解卷积类组。接下来，我们研究了这些标志物的相关性和模式，并利用标准化工作流程系统评估了它们在预测ICB反应和生存状态方面的表现，涵盖了不同的数据集、癌症类型、抗体、活检时间和联合治疗。在我们的基准中，大多数标志物在不同数据集中的稳定性和鲁棒性方面表现不佳。两个评分（TIDE和CYT）在ICB反应预测方面性能竞争激烈，而另外两个评分（PASS-ON和EIGS_ssGSEA）与临床结果关联性最强。最后，我们开发了ICB-Portal来托管这些数据集、标志物和基准结果，并实施计算方法，供研究人员测试他们的自定义标志物。我们的工作提供了有价值的资源和一站式解决方案，以促进与ICB相关的研究。

Immune checkpoint blockades (ICBs) have revolutionized cancer therapy by inducing durable clinical responses, but only a small percentage of patients can benefit from ICB treatments. Many studies have established various biomarkers to predict ICB responses. However, different biomarkers were found with diverse performances in practice, and a timely and unbiased assessment has yet to be conducted due to the complexity of ICB-related studies and trials. In this study, we manually curated 29 published datasets with matched transcriptome and clinical data from more than 1400 patients, and uniformly preprocessed these datasets for further analyses. In addition, we collected 39 sets of transcriptomic biomarkers, and based on the nature of the corresponding computational methods, we categorized them into the gene-set-like group (with the self-contained design and the competitive design, respectively) and the deconvolution-like group. Next, we investigated the correlations and patterns of these biomarkers and utilized a standardized workflow to systematically evaluate their performance in predicting ICB responses and survival statuses across different datasets, cancer types, antibodies, biopsy times, and combinatory treatments. In our benchmark, most biomarkers showed poor performance in terms of stability and robustness across different datasets. Two scores (TIDE and CYT) had a competitive performance for ICB response prediction, and two others (PASS-ON and EIGS_ssGSEA) showed the best association with clinical outcome. Finally, we developed ICB-Portal to host the datasets, biomarkers, and benchmark results and to implement the computational methods for researchers to test their custom biomarkers. Our work provided valuable resources and a one-stop solution to facilitate ICB-related research.