Customized Crowds and Active Learning to Improve Classification



Traditional classification algorithms can be limited in their performance when a specific user is targeted. User preferences, e.g. in recommendation systems, constitute a challenge for learning algorithms. Additionally, in recent years user’s interaction through crowdsourcing has drawn significant interest, although its use in learning settings is still underused.

In this work we focus on an active strategy that uses crowd-based non-expert information to appropriately tackle the problem of capturing the drift between user preferences in a recommendation system. The proposed method combines two main ideas: to apply active strategies for adaptation to each user; to implement crowdsourcing to avoid excessive user feedback. A similitude technique is put forward to optimize the choice of the more appropriate similitude-wise crowd, under the guidance of basic user feedback.

The proposed active learning framework allows non-experts classification performed by crowds to be used to define the user profile, mitigating the labeling effort normally requested to the user.

The framework is designed to be generic and suitable to be applied to different scenarios, whilst customizable for each specific user. A case study on humor classification scenario is used to demonstrate experimentally that the approach can improve baseline active results.


Crowdsourcing, Active learning, Classification

Related Project

iCIS - Intelligent Computing in the Internet of Services


Expert Systems With Applications, Elsevier, Vol. 40, #18, pp. 7212-7219, Jay Liebowitz , December 2013


Cited by

Year 2016 : 1 citations

 Ivens Portugal, Paulo Alencar, Donald Cowan, Requirements Engineering for General Recommender Systems, Cornell University Library, arXiv:1511.05262.

Year 2015 : 2 citations

 Mokter Hossain, Crowdsourcing in business and management disciplines: an integrative literature review, Journal of Global Entrepreneurship Research, 5(21), 2015.

 WANG You-wei,LIU Yuan-ning, FENG Li-zhou, ZHU Xiao-dong, A Novel Quick Online Spam Identification Method Based on User Interest Set, Acta Eletronica Sinica 43(10), 2015.