Using Web Security Scanners to Detect Vulnerabilities in Web Services



Although Web services are becoming business-critical components, they are often deployed with critical software bugs that can be maliciously explored. Web vulnerability scanners allow detecting security vulnerabilities in Web services by stressing the service from the point of view of an attacker. However, research and practice show that different scanners have different performance on vulnerabilities detection. In this paper we present an experimental evaluation of security vulnerabilities in 300 publicly available Web services. Four well known vulnerability scanners have been used to identify security flaws in Web services implementations. A large number of vulnerabilities has been observed, which confirms that many services are deployed without proper security testing. Additionally, the differences in the vulnerabilities detected and the high number of false-positives (35% and 40% in two cases) and low coverage (less than 20% for two of the scanners) observed highlight the limitations of Web vulnerability scanners on detecting security vulnerabilities in Web services.

Download from IEEE Xplore


Web Services Security


39th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2009), June 2009

Cited by

Year 2014 : 15 citations

 M. K. N. Durai and K. Priyadharsini, “A Survey on Security Properties and Web Application Scanner,” International Journal of Computer Science and Mobile Computing, vol. 3, no. 10, pp. 517–527, 2014.

 M. Mirjalili, A. Nowroozi, and M. Alidoosti, “A survey on web penetration test,” Advances in Computer Science: an International Journa, vol. 3, no. 6, 2014.

 ??? and ???, “Design and implementation of website information disclosure assessment system,” 2014.

 T. P. Chiem, “A study of penetration testing tools and approaches,” MSc Thesis, Auckland University of Technology, Auckland, New Zealand, 2014.

 P. D. Buck, Q. Shi, and B. Zhou, “Monitoring and Testing Web Services,” 2014.

 R. M. Jnena, “Modern Approach for WEB Applications Vulnerability Analysis,” MSc Thesis, The Islamic University of Gaza, 2013.

 D. Appelt, N. Alshahwan, D. C. Nguyen, and L. Briand, “Black-box SQL Injection Testing,” University of Luxembourg, TR-SnT-2014-1, 2014.

 D. Appelt, N. Alshahwan, and L. Briand, “Assessing the Impact of Firewalls and Database Proxies on SQL Injection Testing,” in Future Internet Testing, T. E. J. Vos, K. Lakhotia, and S. Bauersfeld, Eds. Springer International Publishing, 2014, pp. 32–47.

 M. Kranch and J. Bonneau, “Upgrading HTTPS in mid-air: An empirical study of strict transport security and key pinning.”

 A. L. Doupé, “Advanced Automated Web Application Vulnerability Analysis,” Ph.D. Thesis, UNIVERSITY OF CALIFORNIA Santa Barbara, Santa Barbara, 2014.

 M. I. P. Salas and E. Martins, “Security Testing Methodology for Vulnerabilities Detection of XSS in Web Services and WS-Security,” in Electronic Notes in Theoretical Computer Science, 2014, vol. 302, pp. 133–154.

 S. Gil, A. Kott, and A.-L. Barabási, “A genetic epidemiology approach to cyber-security,” Sci. Rep., vol. 4, Jul. 2014.

 S. Karumanchi and A. C. Squicciarini, “In the Wild: a Large Scale Study of Web Services Vulnerabilities,” presented at the 29th Symposium On Applied Computing, Gyeongju, Republic of Korea, 2014.

 R. J. Manoj, A. Chandrasekhar, and M. A. Praveena, “An Approach to Detect and Prevent Tautology Type SQL Injection in Web Service Based on XSchema validation,” International Journal Of Engineering And Computer Science, vol. 3, no. 1, pp. 3695–3699, Jan. 2014.

 I. Medeiros, N. F. Neves, and M. Correia, “Automatic Detection and Correction of Web Application Vulnerabilities using Data Mining to Predict False Positives,” in Proceedings of the International World Wide Web Conference (WWW), Seoul, Korea, 2014.

Year 2013 : 13 citations

 ???, ???, and ???, “?? RESTful API ? SQL ??????????????,” ???????????, vol. 30, no. 3, pp. 417–424, 2013.

 P. Zech, M. Felderer, M. Farwick, and R. Breu, “A Concept for Language-Oriented Security Testing,” in 2013 IEEE 7th International Conference on Software Security and Reliability-Companion (SERE-C), 2013, pp. 53–62.

 A. Asmawi, L. S. Affendey, N. I. Udzir, and R. Mahmod, “XIPS: A Model-based Prevention Mechanism for Preventing Blind XPath Injection in Database-Centric Web Services Environment,” International Journal of Advancements in Computing Technology (IJACT), vol. 5, no. 10, 2013.

 L. Stage, “Entwurf einer Methodik zum Testen der Sicherheit von Web-Service-basierten Systemen,” University of Stuttgart, 2013.

 C. Ma, Y. Duan, X. Ju, and F. Xu, “WS-S Evaluation Based on User Preferences and Ranking Mechanism,” in 2013 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2013, pp. 298–301.

 Y.-C. Cho and J.-Y. Pan, “Vulnerability Assessment of IPv6 Websites to SQL Injection and other Application Level Attacks,” The Scientific World Journal, vol. 2013, 2013.

 R. I. Hassan and N. H. B. M. Alwi, “Penetration Testing for Libyan Government Website,” presented at the Fourth International Conference on Computing and Informatics 2013 (ICOCI 2013), Kuching, Sarawak, Malaysia, 2013.

 J. E. Stein, “Metodologia de Configuração de Vulnerabilidades para o Modsecurity,” Colégio Técnico Industrial de Santa Maria - Universidade Federal de Santa Maria, Santa Maria, RS, Brasil, Curso Superior de Tecnologia em Redes de Computadores, 2013.

 N. F. Awang and A. A. Manaf, “Self Assessment Framework For Detecting Vulnerability In Web Applications,” in The Third International Conference on Digital Information and Communication Technology and its Applications (DICTAP2013), 2013, pp. 283–287.

 I. Medeiros, N. F. Neves, and M. Correia, “Securing Energy Metering Software with Automatic Source Code Correction,” in 11th IEEE International Conference on Industrial Informatics (INDIN 2013).

 Z. Zheng and M. R. Lyu, “Background Review,” in QoS Management of Web Services, Springer Berlin Heidelberg, 2013, pp. 9–17.

 S. Wang, Y. Gong, G. Chen, Q. Sun, and F. Yang, “Service Vulnerability Scanning based on Service-oriented Architecture in Web Service Environments,” Journal of Systems Architecture.

 P. Payet, A. Doupé, C. Kruegel, and G. Vigna, “EARs in the Wild: Large-Scale Analysis of Execution After Redirect Vulnerabilities,” 28th Symposium On Applied Computing, Mar. 2013.

Year 2012 : 16 citations

 A. Asmawi, L. S. Affendey, N. I. Udzir, and R. Mahmod, “Model-based system architecture for preventing XPath injection in database-centric web services environment,” in 7th International Conference on Computing and Convergence Technology (ICCCT 2012), Seoul, South Korea, 2012, pp. 621–625.

 S. S. Venkatraman, “Systematically Enhancing Black-Box Web Vulnerability Scanners,” Master of Science, National University of Singapore Singapore, Singapore, 2012.

 M. P. Salas and E. Martins, “Emulation of Malformed XML Using WSInject for Security Testing Against WS-Security,” presented at the IEEE Latin-American Conference on Communications (LATINCOM), Cuenca, Ecuador, 2012.

 D. N. Swetha and B. S. Kumar, “Protocol Based Approach on Vulnerability Detection Tools of SQLIA along with Monitoring Tools,” International Journal of Computer Science Engineering and Technology (IJCSET), vol. 2, no. 11, pp. 1476–1482, Nov. 2012.

 Z. Zheng, Y. Zhang, and M. Lyu, “Investigating QoS of Real-World Web Services,” IEEE Transactions on Services Computing, vol. PP, no. 99, p. 1, 2012.

 D. Hauzar and J. Kofron, “On Security Analysis of PHP Web Applications,” in Computer Software and Applications Conference Workshops (COMPSACW), 2012 IEEE 36th Annual, 2012, pp. 577 –582.

 S. Katkar Anjali and B. Kulkarni Raj, “Web Vulnerability Detection and Security Mechanism,” International Journal of Soft Computing and Engineering (IJSCE), vol. 2, no. 4, pp. 237–241, Sep. 2012.

 M. P. Salas and E. Martins, “Emulação de Ataques do Tipo XPath Injection para Testes de Web Services usando Injeção de Falhas,” in XIII Workshop de Testes e Tolerância a Falhas, Ouro Preto - MG, Brasil, 2012.

 D. Rocha, D. Kreutz, and R. Turchetti, “A free and extensible tool to detect vulnerabilities in Web systems,” in 2012 7th Iberian Conference on Information Systems and Technologies (CISTI), 2012, pp. 1 –6.

 A. Malhotra, N. Navdeep, and G. S. Sekhon, “Browser Prevention Against Phishing Website Security Risk,” International journal of Computer Science & Communication, vol. III, no. 1, pp. 215–219, Jun. 2012.

 WANG Li-Jie, LI Meng, CAI Si-Bo, LI Ge, XIE Bing, and YANG Fu-Qing, “Internet Information Search Based Approach to Enriching Textual Descriptions for Public Web Services,” Journal of Software, vol. 23, no. 6, 2012.

 A. Doupé, L. Cavedon, C. Kruegel, and G. Vigna, “Enemy of the State: A State-Aware Black-Box Web Vulnerability Scanner,” in 21st USENIX Security Symposium (USENIX Security ’12), Bellevue, WA, 2012.

 P. Zech, M. Felderer, and R. Breu, “Towards Risk–Driven Security Testing of Service Centric Systems,” presented at the QSIC, 2012.

 M. H. Al-Ibrahim, “Are Our Educational Technology Systems Secure?,” International Journal of Innovation, Management and Technology (IJIMT), vol. 3, no. 3, pp. 241–245, 2012.

 M. Murali and R. Srinivasan, “Inter-domain Authentication Scheme in a Distributed Mobile Network,” Research Journal of Information Technology, 2012.

 V. Prasath, “Building Trust for Web Services Security Patterns,” International Journal of Applied Information Systems (IJAIS), vol. 3, no. 2, pp. 14–20, Jul. 2012.

Year 2011 : 9 citations

 D. Hauzar and J. Kofron, “Hunting Bugs Inside Web Applications,” Technical Report, Oct. 2011.

 SU Bin and YANG Yin, “The Limitations of Network Applications Vulnerability Scanner,” Network and Computer Security, no. 5, pp. 77–79, 2011.

 F. van der Loo, “Comparison of penetration testing tools for web applications,” MSc Thesis, University of Radboud, Netherlands, 2011.

 V. Shanmughaneethi, R. Ravichandran, and S. Swamynathan, “PXpathV: Preventing XPath Injection Vulnerabilities in Web Applications,” International Journal on Web Service Computing, vol. 2, no. 3, pp. 192–201, Sep. 2011.

 A. M. Ferreira and H. Kleppe, “Effectiveness of Automated Application Penetration Testing Tools,” 2011.

 J. Chen and S. Kulkarni, “Effectiveness of Transition Systems to Model Faults,” in Proceedings of the 2nd International Workshop on Logical Aspects of Fault-Tolerance (LAFT) In conjunction with LICS’11., Toronto, Canada, 2011.

 V. Shanmughaneethi, R. Y. Pravin, and S. Swamynathan, “XIVD: Runtime Detection of XPath Injection Vulnerabilities in XML Databases through Aspect Oriented Programming,” Advances in Computing and Information Technology, pp. 192–201, 2011.

 Z. Zheng, “QoS Management of Web Services,” Ph.D. Thesis, The Chinese University of Hong Kong, Hong Kong, 2011.

 Wang Xin, Wei Gengyu, Zhang Dongmei, Yang Yixian, "Web Application Vulnerability Detection Based on Reinforcement Learning", 3rd International Conference on Computer and Network Technology, ICCNT 2011, Taiyuan, China, February 26-28, 2011.

Year 2010 : 10 citations

 AOKI T. and YASUDA H., “Web Fingerprint: A New Scheme to Arbitrate Mismatch of Web Pages,” The Journal of the Institute of Image Electronics Engineers of Japan, vol. 39, no. 5, pp. 644–653, Sep. 2010.

 X. Wang, L. Wang, G. Wei, D. Zhang, and Y. Yang, “Hidden web crawling for SQL injection detection,” in 3rd IEEE International Conference on Broadband Network and Multimedia Technology (IC-BNMT), Beijing, China, 2010, pp. 14–18.

 M. P. Correia and P. J. Sousa, Segurança no Software. Lisboa, Portugal: FCA, 2010.

 C. Lai-Cheng, “Enhancing Distributed Web Security Based on Kerberos Authentication Service,” in Web Information Systems and Mining, vol. 6318, F. Wang, Z. Gong, X. Luo, and J. Lei, Eds. Springer Berlin / Heidelberg, 2010, pp. 171–178.

 D. A. Shelly, “Using a Web Server Test Bed to Analyze the Limitations of Web Application Vulnerability Scanners,” MSc Thesis - Master of Science in Computer Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, 2010.

 T. Basso, R. L. O. Moraes, and M. Jino, “A Methodology for Effectiveness Analysis of Vulnerability Scanning Tools,” presented at the Terceiro Encontro dos Alunos e Docentes do Departamento de Engenharia de Computação e Automação Industrial, University of Campinas (UNICAMP), Brazil, 2010.

 T. Basso, P. C. S. Fernandes, M. Jino, and R. L. O. Moraes, “Analysis of the Effect of Java Software Faults on Security Vulnerabilities and Their Detection by Commercial Web Vulnerability Scanner Tool,” in 4th Workshop on Recent Advances on Intrusion-Tolerant Systems, WRAITS 2010, in conjunction with The 40th IEEE/IFIP International Conference on Dependable Systems and Networks, Chicago, IL, USA, 2010.

 A. Doupé, M. Cova, and G. Vigna, “Why Johnny Can’t Pentest: An Analysis of Black-Box Web Vulnerability Scanners,” Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 111–131, 2010.

 Z. Zheng, Y. Zhang, and M. R. Lyu, “Distributed QoS Evaluation for Real-World Web Services,” in 2010 IEEE International Conference on Web Services, 2010, pp. 83–90.

 L. Wang, F. Liu, L. Zhang, G. Li, and B. Xie, “Enriching Descriptions for Public Web Services Using Information Captured from Related Web Pages on the Internet,” in 2010 Fifth IEEE International Symposium on Service Oriented System Engineering, 2010, pp. 141–150.