{ "abstract": "Methods for segmentation of skull infiltrated tumors in Computed\r\nTomography (CT) images using Insight Segmentation and Registration\r\nToolkit ITK (www.itk.org) are presented. Pipelines of filters and\r\nalgorithms from ITK are validated on the basis of different criteria: sensitivity,\r\nspecificity, dice similarity coefficient, Chi-squared, and Hausdorff\r\ndistance measure. The method to rate segmentation results in relation\r\nto validation metrics is presented together with analysis of importance of\r\ndifferent goodness measures. Results for one simulated dataset and three\r\npatient are presented.", "authors": [ { "author_fullname": "Popovic, Aleksandra", "author_place": 1, "persona_email": "popovic@hia.rwth-aachen.de", "persona_firstname": "Aleksandra", "persona_id": 50, "persona_lastname": "Popovic" }, { "author_fullname": "Engelhardt, Martin", "author_place": 2, "persona_id": null }, { "author_fullname": "Radermacher, Klaus", "author_place": 3, "persona_id": null } ], "categories": [], "comments": [], "date_submitted": "2005-08-01T16:29:18Z", "journals": [ { "journal_id": 3, "journal_name": "The Insight Journal" } ], "license": "You are licensing your work to Kitware Inc. under the\nCreative Commons Attribution License Version 3.0.\n\nKitware Inc. agrees to the following:\n\nKitware is free\n * to copy, distribute, display, and perform the work\n * to make derivative works\n * to make commercial use of the work\n\nUnder the following conditions:\n\\\"by Attribution\\\" - Kitware must attribute the work in the manner specified by the author or licensor.\n\n * For any reuse or distribution, they must make clear to others the license terms of this work.\n * Any of these conditions can be waived if they get permission from the copyright holder.\n\nYour fair use and other rights are in no way affected by the above.\n\nThis is a human-readable summary of the Legal Code (the full license) available at\nhttp://creativecommons.org/licenses/by/3.0/legalcode", "publication_id": 17, "reviews": [ { "author": { "author_email": "luis.ibanez@kitware.com", "author_firstname": "Luis", "author_id": 4, "author_lastname": "Ibanez" }, "content": "Summary:\r\nThis papers describes the application of algorithms available in ITK to the segmentation of brain-tumors infiltrating the skull. It also implements a methodology for evaluating their results compared to a human expert.\r\n \r\nHypothesis:\r\nThis paper hypothesis that segmentation methods from the Insight Toolkit can be used for delineating skull infiltrated tumors. In order to test this hypothesis, the authors selected multiple segmentation methods from the toolkit, ran segmentation on test data and implemented an evaluation methodology for comparing the results from the software-based segmentation with the human expert segmentations.\r\n\r\nEvidence:\r\nThe authors present abundant evidence from the result of their segmentation experiments. The comparision of the segmentation methods reveals that the authors invested a significan amount of work in exploring the applicability of the many segmentation methods compared here. It is unfortunate that the full material of those experiments was not shared with the readers, who would have derived great benefits from being able to apply the same set of experiments to other medical segmentation problems.\r\n\r\nOpen Science:\r\nThe paper describe the segmentation methods evaluated in this case, and the evaluation methodology used for comparing the results. This evaluation methodology seems to be the most valuable contribution of the paper, since it reveals that a wide set of segmentation tools were systematically tested. Unfortunately the authors did not share the source code used for their test and for their evaluation methodology. Despite the fact that their code is based on ITK filters, it would take a reader a significant amount of time to replicate the work of the authros, since the reader will have to reimplement the code for all the tests.\r\n\r\nReproducibility:\r\nThe work can hardly be reproduced. Not enough coding details are provided by the authors, and in particular, no details regarding the many numerical parameters of the segmentation methods are mentioned. In the current form it is impossible to replicate the work performed by the authors. A reader could run an equivalent set of experiments, by using ITK filters, but the lack of the input images, and the lack of details regarding the parameters will make impossible to ensure that the reader\\'s set of test is equivalent to the one reported by the authors.\r\n\r\n\r\nUse of Open Source Software:\r\nThe authors make extensive use of the Insight Toolkit, and took care of using ITK filters as building blocks. Their study of the applicability of ITK segmentation filters is certainly valuable for the readers, since it guides them towards the segmentation methods that displayed the best performance (compared to a human expert) for segmenting the skull-infiltrated tumors.\r\n\r\n\r\n\r\nOpen Source Contributions:\r\nThe authors did not provide their source code.\r\n\r\n\r\nCode Quality:\r\nReviewer\\'s speculation: Judgging from the description in the paper it seems that the authors made a good use of the code available in the Insight Toolkit, by combining ITK filters as building blocks for their segmenation pipelines. \r\n\r\nApplicability to other problems:\r\nThe generic problem faced by the authors is indeed quite challenging since the intensity of the image alone is not enough for segmenting the objets. The ITK methods that the authors found to be more appropriate for this problem, will probably perform well for other segmentation situations where the intensity of the object to be extracted is similar to the intensity of adjacent structures.\r\n\r\n\r\nSuggestions for future work:\r\nMany of the methods used in VALMET (Gerig, G., Jomier, M., Chakos, M.) are now available in ITK in the form of filters. In particular the overlap measures, the Haussdorf distance and surface mean distance. It could be interesting to create a reusable framework for segmentation validation that any authors could use during the validation stages of their segmentation work.\r\n\r\n\r\n\r\nRequests for additional information from authors:\r\nIt will be very useful if the authors share their source code with the community. In particular all the parameters setting of the segmentation methods. The source code of their evaluation framework will also be extremely useful for anybody performing a segmentation evaluation study.\r\n\r\n\r\nAdditional Comments:\r\nSince the authors found the Chi squared value to be a convenient measure for comparing the segmentation results to the human expert delineation, it seems to be interesting to create that measure as an ITK filter.\r\n\r\n\r\n", "date": "09-18-2005", "review_id": 105 }, { "author": { "author_email": "sylvain@bwh.harvard.edu", "author_firstname": "Sylvain", "author_id": 87, "author_lastname": "Bouix" }, "content": "Summary:\r\nThis paper describes a validation study of the segmentation of skull infiltrated tumors.\r\nTwo segmentation pipelines implemented in ITK are presented, one based on region growing, one based on level sets.\r\nThey are then validated agains one expert\\\\\\'s segmentations using five similarity measurement: sensitivity, specificity, Dice, X2 and Hausdorff\r\n\r\nHypothesis:\r\nNon applicable\r\n\r\nEvidence:\r\nFour datasets were used, 3 patients with manual tracings by 1 human expert and 1 phantom with ground truth.\r\nThe experimetns are clearly explained and sufficient evidence is provided.\r\n\r\nOpen Science:\r\nThere is enough information in the paper to reproduce the experiment, but no data or source code is provided. \r\n\r\nUse of Open Source Software:\r\nAuthors used ITK for the segmentation pipelines and Hausdorff measurements\r\n\r\nOpen Source Contributions:\r\nNo code provided.\r\n\r\nApplicability to other problems:\r\nThe main contribution of the paper is to question certain measurement when validating segmentation technique, which I think is important for the community in general. They partiucularly caution the reader in using specificity and sensitivity as the sole validation parameters as they are dependent on the image size vs. object size relationship. They later show that these measures actually rank region growing higher than level sets even though other measures disagree. \r\n\r\nSuggestions for future work :\r\nIn the context of open science, I suggest the author incorporate the validation measurements into ITK and share their data and source code so experiments can be reproduced elsewhere.\r\nI also suggest they investigate other similarity measures found in the statistic litterature. The article by Hripcsakk abd Heitjjan on \\\\\\\"Measuring Agreement in medical informatics reliability studies\\\\\\\" is a good starting point. \r\n\r\n", "date": "08-24-2005", "review_id": 38 } ], "revisions": [ { "article": "bafkreic3rq3hvggq6tecga2pqur7xpt67v3uc3juv6tjeqt26pnvs7hihy", "citation_list": [ { "doi": "10.1016/s0531-5131(03)00398-4", "key": "ref1", "score": 132.9283, "unstructured": "CRANIO - Computer Assisted Planning for Navigation and Robot-assisted Surgery on the Skull+: Proceedings of the 17th International Congress and Exhibition (CARS)+2003+1269+1276+A. Popovic+M. Engelhardt+T. Wu+F. Portheine+K. Schmieder+K. Radermacher+M. Vannier+K. Inamura+A. Farman+K. Doi+Reiber+Elsevier" }, { "doi": "10.3171/jns.2000.93.6.0940", "key": "ref2", "score": 116.3627, "unstructured": "Primary Extradural Meningiomas: A Report of Nine Cases and Review of the Literature from the Era of Computerized Tomography Scanning+J Neurosurg+93+2000+940+950+F.F. Lang+O. Macdonald+G. Fuller+DeMonte" }, { "doi": "10.1007/s00234-003-1070-5", "key": "ref3", "score": 124.72109, "unstructured": "The CT Triad of Malignacy in Meningioma - Redefinition, with a Report of Three New Cases. Neuroradiology+45+2003+799+803+S. Grover+A. Aggarwal+P.S. Uppal+R. Tandon" }, { "key": "ref4", "score": 53.086224, "unstructured": "Neurosurg Rev+26+2003+73+99+P. Grunert+J. anf+K.D. Espinosa+Filippi" }, { "doi": "10.1007/3-540-45468-3_62", "key": "ref5", "score": 174.38818, "unstructured": "Valmet: A New Validation Tool for Assessing and Improving 3d Object Segmentation+In: MICCAI 2001: Fourth International Conference on Medical Image Computing and Computer-Assisted Intervention. Volume 2208 of Lecture Notes in Computer Science+2001+516+528+G. Gerig+M. Jomier+M. Chakos" }, { "doi": "10.1201/b10657", "key": "ref6", "score": 74.72027, "unstructured": "Insight into Images. Principles and Practice for Segmentation, Registration, and Image Analysis. A K Peters+2004+Yoo" }, { "key": "ref7", "score": 40.593365, "unstructured": "The ITK Software Guide. 2 edn. ITK Software Consortium+2005+L. Ibanez+W. Schroeder+L.N.J. Cates" }, { "doi": "10.1109/tmi.2004.828354", "key": "ref8", "score": 122.95451, "unstructured": "Simultaneous Truth and Performance Level Estimation (staple): An Algorithm for the Validation of Image Segmentation+IEEE Trans. on Medical Imaging+23+2004+903+921+K.H.Z. Warfield+W.M. Wells" }, { "doi": "10.1016/s1076-6332(03)00671-8", "key": "ref9", "score": 126.45005, "unstructured": "Statistical Validation of Image Segmntation Quality Based on a Spatial Overlap Index+Acad Radiol+11+2004+178+189+K.H. Zou+S.K. Warfield+A. Bharatha+C.M. Tempany+M.R. Kaus+S.J. Haker+W.M. Wells+F.A. Jolesz+K. Ron" }, { "doi": "10.1109/tpami.2003.1217608", "key": "ref10", "score": 113.72703, "unstructured": "A Method for Objective Edge Detection Evaluation and Detector Parameter Selection+IEEE Trans. on Pattern Analysis and Machine Intelligence+25+2003+1+7+Y. Yitzhaky+Peli" } ], "dapp": null, "dataset": null, "doi": "10.54294/q9va9k", "handle": "1926/21", "source_code": null, "source_code_git_ref": null } ], "source_code_git_repo": null, "submitted_by_author": { "author_email": "popovic@hia.rwth-aachen.de", "author_firstname": "Aleksandra", "author_fullname": "Popovic, Aleksandra", "author_id": 50, "author_institution": "Helmholtz Institute for Biomedical Engineerning, RWTH Aachen University", "author_lastname": "Popovic" }, "tags": [ "ITK", "Region Growing", "Level Sets", "Validation", "Segmentation" ], "title": "Segmentation of Skull-infiltrated Tumors Using ITK: Methods and Validation" }