Macrophage+: A game with a purpose for applying human intelligence in control mechanisms

Document Type : Article

Authors

Department of Computer Science and Engineering, Shahid Beheshti University G. C., Tehran, Iran

Abstract

Originally, control mechanisms were proposed to replace the need for human intervention in operational environments, and thus, enhance the precision and reaction time. Nowadays, new requirements in computer systems such as adaptation have made the design of control mechanisms more challenging. The Observer/Controller pattern is one of the control mechanisms proposed to control many interacting independent elements by making intelligent decisions. An important challenge in designing these mechanisms is that the knowledge needed for decision making is provided by experts; therefore, the process becomes time consuming and costly, depending on the availability of experts and their costs. In this paper, we hypothesize that employing a Game With A Purpose can help to improve the process of providing knowledge in such control mechanisms by using crowd-sourcing and involving non-expert humans in an enjoyable manner. This hypothesis has been investigated by Macrophage+, a Game With A Purpose implemented for this goal. We conducted experiments evaluating Macrophage+, focusing on both its applicability and effectiveness in the context of the observer/controller pattern as well as its enjoyability for the players. The results show that Macrophage+ is a successful Game With A Purpose that involves non-expert humans in the application of the observer/controller pattern.

Keywords


References
1. Shaw, M., Andersson, J., Litoiu, M., Schmerl, B.,
Tamura, G., Villegas, N.M., Vogel, T., Weyns, D.,
Baresi, L., Becker, B., Bencomo, N., Brun, Y., Cukic,
B., Desmarais, R., Dustdar, S., Engels, G., Geihs, K.,
Goschka, K.M., Gorla, A., Grassi, V., Inverardi, P.,
Karsai, G., Kramer, J., Lopes, A., Magee, J., Malek,
S., Mankovskii, S., Mirandola, R., Mylopoulos, J.,
Nierstrasz, O., Pezze, M., Prehofer, C., Schafer, W.,
Schlichting, R., Smith, D.B., Sousa, J.P., Tahvildari,
L., Wong, K., andWuttke, J. Software engineering for
selfadaptive systems: A second research Roadmap",
De Lemos, R., Giese, H., Muller, H.A., Shaw, M.,
Editors, In Software Engineering for Self-Adaptive
Systems II. Lecture Notes in Computer Science, 7475,
pp. 1{32, Springer, Berlin, Germany (2013).
2. Nafz, F., Steghofer, J.P., Seebach, H., et al. Formal
modeling and veri cation of self-* systems based on
observer/controller architectures", In Camara J., de
Lemos R., Ghezzi, C., and Lopes A., Editors, Assurances
for Self-Adaptive Systems, Lecture Notes in
Computer Science, 7740, pp. 80{111, Springer (2013).
3. Salehie, M. and Tahvildari, L. Self-adaptive software:
Landscape and research challenges", Trans. Auton.
Adapt. Syst., 4(2), pp. 1{42 (2009).
4. Macas-Escriva, F.D., Haber, R., del Toro, R., et
al. Self-adaptive systems: A survey of current approaches,
research challenges and applications", Expert
Systems with Applications, 40(18), pp. 7267{7279
(2013).
5. Kephart, J.O. and Chess, D.M. The vision of autonomic
computing", Computer, 36(1), pp. 41{50
(2003).
6. Muller-Schloer, C. and Tomforde, S., Organic Computing
- Technical Systems for Survival in the Real World,
Springer, Berlin, Germany (2017).
7. Muller-Schloer, C., Schmeck, H., and Ungerer, T.,
Editors, Organic Computing - A paradigm Shift for
Complex Systems, 1st Edn., Springer, Berlin, Germany
(2011).
8. Seebach, H., Ortmeier, F., and Reif, W. Design
and construction of organic computing systems", In
Evolutionary Computation, CEC 2007, pp. 4215{4221,
Singapore (2007).
9. Krupitzer, C., Roth, F.M., VanSyckel, S., et al. A
survey on engineering approaches for self-adaptive
systems", Pervasive and Mobile Computing, 17, pp.
184{206 (2015).
10. Momtazi, S. and Moradiannasab, O. A statistical
approach to knowledge discovery: Bootstrap analysis
of language models for knowledge base population from
unstructured text", Scientia Iranica, 26, pp. 26{39
(2019).
11. Winter, S., Richter, K.F., Baldwin, T., et al.
Location-based mobile games for spatial knowledge
acquisition", In Proc. of the Workshop on Cognitive
Engineering for Mobile GIS, Belfast, Ireland (2011).
12. Ponsen, M., Spronck, P., Munoz-Avila, H., et al.
Knowledge acquisition for adaptive game AI", Science
of Computer Programming, 67(1), pp. 59{75 (2007).
A. Tarihi et al./Scientia Iranica, Transactions D: Computer Science & ... 27 (2020) 2985{3004 3001
13. Bimba, A.T., Idris, N., Al-Hunaiyyan, A., et al.
Towards knowledge modeling and manipulation technologies:
A survey", Intl. Journal of Information
Management, 36(6), pp. 857{871 (2016).
14. Von Ahn, L. Games with a purpose", Computer,
39(6), pp. 92{94 (2006).
15. Cooper, S., Khatib, F., Treuille, A., et al. Predicting
protein structures with a multiplayer online game",
Nature, 466(7307), pp. 756{760 (2010).
16. Law, E.L.M., Von Ahn, L., Dannenberg, R.B., et al.
Tagatune: A game for music and sound annotation",
In Proc. 8th Intl. Conf. on Music Information Retrieval
(ISMIR), 3, pp. 361{364, Vienna, Austria (2007).
17. Quinn, A.J. and Bederson, B.B. Human computation:
A survey and taxonomy of a growing eld", In Proc.
of the SIGCHI Conf. on Human Factors in Computing
Systems (CH'11), pp. 1403{1412, New York, USA
(2011).
18. Sabou, M., Bontcheva, K., Scharl, A., et al. Games
with a purpose or mechanized labor?: A comparative
study", In Proc. of the 13th Intl. Conf. on Knowledge
Management and Knowledge Technologies, p. 19, Graz,
Austria (2013).
19. Wang, A., Hoang, C.D.V., and Kan, M.K. Perspectives
on crowdsourcing annotations for natural
language processing", Language Resources and Evaluation,
47(1), pp. 9{31 (2013).
20. Weng, L., Schifanella, R., and Menczer, F. Design
of social games for collecting reliable semantic annotations",
In 16th Intl. Conf. on Computer Games
(CGAMES), pp. 185{192, Louisville, KY, USA (2011).
21. Reason, J., Human Error, Cambridge University Press
(1990).
22. Rasmussen, J. Skills, rules, and knowledge; signals,
signs, and symbols, and other distinctions in human
performance models", Systems, Man and Cybernetics,
IEEE, Transactions on, 3, pp. 257{266 (1987).
23. Lalanda, P., McCann, J.A., and Diaconescu, A.
Autonomic computing", In Principles, Design and
Implementation, 1st Edn., Springer, Germany (2013).
24. Garlan, D., Schmerl, B., and Cheng, S.W. Software
architecture-based self-adaptation", In Autonomic
Computing and Networking, pp. 31{55, Springer, Germany
(2009).
25. Garlan, D., Cheng, S.W., Huang, A.C., et al. Rainbow:
Architecture-based self-adaptation with reusable
infrastructure", Computer, 37(10), pp. 46{54 (2004).
26. Oreizy, P., Medvidovic, N., and Taylor, R.N.
Architecture-based runtime software evolution", In
Proc. of the 20th Intl. Conf. on Software Engineering,
pp. 177{186, Kyoto, Japan (1998).
27. Canal, C., Poizat, P., and Salaun, G. Model-based
adaptation of behavioral mismatching components",
IEEE Trans. on Software Engineering, 34(4), pp. 546{
563 (2008).
28. Filieri, A., Ho mann, H., and Maggio, M. Automated
design of self-adaptive software with control theoretical
formal guarantees", In Proc. of the 36th Intl. Conf. on
Software Engineering, pp. 299{310, Hyderabad, India
(2014).
29. Nafz, F., Ortmeier, F., Seebach, H., et al. A generic
software framework for role-based organic computing
systems", In Software Engineering for Adaptive and
Self-Managing Systems, SEAMS'09. ICSE Workshop
on, pp. 96{105, Cape Town, South Africa (2009).
30. Fischer, P., Nafz, F., Seebach, H., et al. Ensuring
correct self-recon guration in safety-critical applications
by veri ed result checking", In Proc. of the 2011
Workshop on Organic Computing, OC'11, pp. 3{12,
New York, USA (2011).
31. Kaiser, G., Parekh, J., Gross, P., et al. Kinesthetics
extreme: an external infrastructure for monitoring
distributed legacy systems", In Autonomic Computing
Workshop, pp. 22{30, Seattle, Washington, USA
(2003).
32. Georgas, J.C. and Taylor, R.N. Towards a knowledgebased
approach to architectural adaptation management",
In Proc. of the 1st ACM SIGSOFT Workshop
on Self-managed Systems, pp. 59{63, Newport Beach,
California, USA (2004).
33. Fox, K.L., Henning, R.R., Farrell, J.T., et al. System
and method for assessing the security posture of a
network using goal oriented fuzzy logic decision rules",
April 19 2005. US Patent 6,883,101 (2005).
34. Berry, D.M., Cheng, B.H.C., and Zhang, J. The
four levels of requirements engineering for and in
dynamic adaptive systems", In 11th Intl. Workshop
on Requirements Engineering Foundation for Software
Quality (REFSQ'05), p. 5, Porto, Portugal (2005).
35. Cheng, B.H.C., Sawyer, P., Bencomo, N., et al. A
goal-based modeling approach to develop requirements
of an adaptive system with environmental uncertainty",
In Intl. Conf. on Model Driven Engineering
Languages and Systems, pp. 468{483, Denver, Colorado,
USA (2009).
36. Van Lamsweerde, A. and Letier, E. Handling obstacles
in goal-oriented requirements engineering", IEEE
Transactions on Software Engineering, 26(10), pp.
978{1005 (2000).
37. Sykes, D., Heaven, W., Magee, J., et al. From goals
to components: a combined approach to selfmanagement",
In Proc. of the 2008 Intl. Workshop on Software
Engineering for Adaptive and Selfmanaging Systems,
pp. 1{8, Leipzig, Germany (2008).
38. Menasce, D.A., Ewing, J.M., Gomaa, H., et al. A
framework for utility-based service oriented design in
sassy", In Proc. of the First Joint WOSP/SIPEW Intl.
Conf. on Performance Engineering, pp. 27{36, San
Jose, California, USA (2010).
3002 A. Tarihi et al./Scientia Iranica, Transactions D: Computer Science & ... 27 (2020) 2985{3004
39. Shen, C.C., Srisathapornphat, C., and Jaikaeo, C.
An adaptive management architecture for ad hoc
networks", IEEE Communications Magazine, 41(2),
pp. 108{115 (2003).
40. Esfahani, N., Elkhodary, A., and Malek, S. A
learning-based framework for engineering featureoriented
self-adaptive software systems", IEEE Transaction
on Software Engineering, 39(11), pp. 1467{1493
(2013).
41. Brinkschulte, U., Pacher, M., and Von Renteln, A. An
arti cial hormone system for self-organizing realtime
task allocation in organic middleware", In Organic
Computing, pp. 261{283, Springer, Berlin, Germany
(2009).
42. Lodish, H., Berk, A., Kaiser, C.A., Krieger, M.,
Bretscher, A., Ploegh, H., Amon, A., Martin, K.C.,
Molecular Cell Biology, 8th Edn, W.H. Freeman and
Company, New York, USA (2008).
43. Alberts, B., Bray, D., Hopkin, K., Johnson, A.D.,
Lewis, J., Ra , M., Roberts, K., Walter, P., Essential
Cell Biology, 4th Edn, Garland Science, Taylor &
Francis Group, New York, USA (2013).
44. Sugeno, M. and Nishida, M. Fuzzy control of model
car", Fuzzy Sets Systems, 16(2), pp. 103{113 (1985).
45. Schmeck, H., Muller-Schloer, C., Cakar, E., et al.
Adaptivity and self-organization in organic computing
systems", ACM Trans. Auton. Adapt. Syst., 5(3),
pp. 1{10 (2010).
46. Richter, U., Mnif, M., Branke, J., et al. Towards
a generic observer/controller architecture for organic
computing", GI Jahrestagung, 93, pp. 112{119 (2006).
47. Komann, M. and Fey, D. Marching pixels-using
organic computing principles in embedded parallel
hardware", In Intl. Symposium on Parallel Computing
in Electrical Engineering (PARELEC'06), pp. 369{
373, Bialystok, Poland (2006).
48. Hartmann, J., Stechele, W., and Maehle, E. Selfadaptation
for mobile robot algorithms using organic
computing principles", In Kubatova, H., Hochberger,
C., Danek M., Sick B., Editors, In Intl. Conf. on
Architecture of Computing Systems (ARCS'13), pp.
232{243, Lecture Notes in Computer Science, 7767,
Springer, Berlin, Germany (2013).
49. Branke, J., Mnif, M., Muller-Schloer, C., et al.
Organic computing-addressing complexity by controlled
self-organization", In Second Intl. Symposium
on Leveraging Applications of Formal Methods, Veri
cation and Validation (Isola 2006), pp. 185{191,
Paphos, Cyprus (2006).
50. Roth, M., Schmitt, J., Kiefhaber, R., et al. Organic
computing middleware for ubiquitous environments",
In Organic Computing A Paradigm Shift for Complex
Systems, pp. 339{351, Springer, Berlin, Germany
(2011).
51. Murch, R., Autonomic Computing. Information Management,
IBM Press, California, USA (2004).
52. Gudemann, M., Nafz, F., Ortmeier, F., et al. A
speci cation and construction paradigm for organic
computing systems", In Second IEEE Intl. Conf. on
Self-Adaptive and Self Organizing Systems, pp. 233{
242, Venezia, Italy (2008).
53. Kasinger, H. and Bauer, B. Combining multi-agentsystem
methodologies for organic computing systems",
In 16th Intl. Workshop on Database and Expert Systems
Applications (DEXA'05), pp. 160{164, Copenhagen,
Denmark (2005).
54. Tomforde, S., Rudolph, S., Bellman, K., et al. An organic
computing perspective on self-improving system
interweaving at runtime", In IEEE Intl. Conf. on Autonomic
Computing (ICAC), pp. 276{284, Wurzburg,
Germany (2016).
55. Herzig, P., Ameling, M., and Schill, A. A generic
platform for enterprise gami cation", In Joint Working
IEEE/IFIP Conf. on Software Architecture and European
Conf. on Software Architecture, pp. 219{223,
Helsinki, Finland (2012).
56. Khanzadia, M., Shahbazia, M.M., Arashpourb, M.,
et al. Lean design management using a gami ed
system", Scientia Iranica, 26(1), pp. 15{25 (2019).
57. Ido, G., Hashavit, A. and Corem, Y. Games for
crowds: A crowd sourcing game platform for the
enterprise", In Proc. of the 18th ACM Conf. on Computer
Supported Cooperative Work and Social Computing,
CSCW'15, pp. 1860{1871, Vancouver, British
Columbia, Canada (2015).
58. Rani, P., Sarkar, N., and Liu, C. Maintaining optimal
challenge in computer games through real-time physiological
feedback", In Proc. of the 11th Intl. Conf. on
Human Computer Interaction, 58, pp. 22{27, Bonn,
Germany (2005).
59. Walsh, G. and Golbeck, J. Curator: A game with a
purpose for collection recommendation", In Proc. of
the SIGCHI Conf. on Human Factors in Computing
Systems, pp. 2079{2082, Atlanta, Gorgia, USA (2010).
60. Chen, L.J., Wang, B.C., and Zhu, W.Y. The design of
puzzle selection strategies for esp-like gwap systems",
IEEE Trans. on Computational Intelligence and AI in
Games, 2(2), pp. 120{130 (2010).
61. Kerler, S., Vilsmeier, J., Edenhofer, S., et al. Pheromander:
Real-time strategy with digital pheromones",
In 8th Intl. Conf. on Games and Virtual Worlds for Serious
Applications (VS-GAMES), pp. 1{4, Wurzburg,
Germany (2016).
62. Ho, C.J., Chang, T.H., Lee, J.C., et al. Kisskissban:
A competitive human computation game for image
annotation", ACM SIGKDD Explorations Newsletter,
12(1), pp. 21{24 (2010).
63. Syu, Y.S., Yu, H.H., and Chen, L.J. Exploiting puzzle
diversity in puzzle selection for esp-like gwap systems",
In IEEE/WIC/ACM Intl. Conf. on Web Intelligence
and Intelligent Agent Technology, 1, pp. 468{475,
Washington, District of Columbia, USA (2010).
A. Tarihi et al./Scientia Iranica, Transactions D: Computer Science & ... 27 (2020) 2985{3004 3003
64. Otterbacher, J. Crowdsourcing stereotypes: Linguistic
bias in metadata generated via gwap", In Proc.
of the 33rd Annual ACM Conf. on Human Factors in
Computing Systems, pp. 1955{1964, Seoul, Republic of
Korea (2015).
65. Harris, C. Cluemein: Enhancing the esp game to
obtain more speci c image labels", In Proc. of the 2018
Annual Symposium on Computer-Human Interaction
in Play Companion Extended Abstracts, CHI PLAY
'18 Extended Abstracts, pp. 447{452, Melbourne, Victoria,
Australia (2018).
66. Ashraf, G., Why, Y.P., and Islam, T. Mining human
shape perception with role playing games", GSTF
Journal on Computing (JoC), 1(1), pp. 58{64 (2018).
67. Amiri-Chimeh, S., Haghighi, H., Vahidi-Asl, M., et
al. Rings: A game with a purpose for test data
generation", Interacting with Computers, 30(1), pp.
1{30 (2017).
68. Kondreddi, S.K., Trianta llou, P., and Weikum, G.
Combining information extraction and human computing
for crowdsourced knowledge acquisition", In
IEEE 30th Intl. Conf. on Data Engineering, pp. 988{
999, Chicago, Illinois, USA (2014).
69. Von Ahn, L., Kedia, M., and Blum, M. Verbosity: A
game for collecting common-sense facts", In Proc. of
the SIGCHI Conf. on Human Factors in Computing
Systems (CHI '06), pp. 75{78, Montreal, Quebec,
Canada (2006).
70. Bellotti, F., Berta, R., De Gloria, A., et al. Exploring
gaming mechanisms to enhance knowledge acquisition
in virtual worlds", In Proc. of the 3rd Intl. Conf. on
Digital Interactive Media in Entertainment and Arts,
pp. 77{84, Athens, Greece (2008).
71. Fisch, D., Janicke, M., Muller-Schloer, C., et al. Divergence
measures as a generalized approach to quantitative
emergence", In Organic Computing-A Paradigm
Shift for Complex Systems, pp. 53{66, Springer Berlin,
Germany (2011).
72. Seebach H., Nafz F., Steghofer J.P., et al. How
to design and implement selforganising resource-
ow
systems", In Organic Computing-A Paradigm Shift
for Complex Systems, pp. 145{161, Springer Berlin,
Germany (2011).
73. Quinlan, J.R. Induction of decision trees", Machine
Learning, 1(1), pp. 81{106 (1986).
74. Kohavi, R. and Quinlan, J.R. Data mining tasks and
methods: Classi cation: decision-tree discovery", In
Handbook of Data Mining and Knowledge Discovery,
pp. 267{276. Oxford University Press, Inc. Oxford,
United Kingdom (2002).
75. Nakamura, S., Nakagawa, H., Tahara, Y., et al.
Towards solving an obstacle problem by the cooperation
of uavs and ugvs", In Proc. of the 28th Annual
ACM Symposium on Applied Computing, pp. 77{82,
Coimbra, Portugal (2013).
76. Venkatesh, S., Lobo, R., and Sugunan, N. Controlling
unmanned ground vehicle using stationary airborne
system", Intl. Journal of Innovation and Applied Studies,
8(3), p. 958 (2014).
77. Grocholsky, B., Keller, J., Kumar, V., et al. Cooperative
air and ground surveillance", IEEE Robotics &
Automation Magazine, 13(3), pp. 16{25 (2006).
78. MacArthur, E.Z., MacArthur, D., et al. Use of cooperative
unmanned air and ground vehicles for detection
and disposal of mines", In Proc. of Intl. Society for
Optics and Photonics, 5999, pp. 94{101 (2005).
79. Liu, Y. and Nejat, G. Robotic urban search and rescue:
A survey from the control perspective", Journal
of Intelligent & Robotic Systems, 72(2), pp. 147{165
(2013).
80. Anderson, B. and Crowell, J. Workhorse auv-a costsensible
new autonomous underwater vehicle for surveys/
soundings, search & rescue and research", In
Proc. of OCEANS 2005 MTS/IEEE, pp. 1{6, Washington,
District of Columbia, USA, IEEE (2005).
81. Mayer, I., Bekebrede, G., Harteveld, C., et al. The
research and evaluation of serious games: Toward
a comprehensive methodology", British Journal of
Educational Technology, 45(3), pp. 502{527 (2014).
82. Brockmyer, J.H., Fox, C.M., Curtiss, K.A., et al. The
development of the game engagement questionnaire:
A measure of engagement in video game-playing",
Journal of Experimental Social Psychology, 45(4), pp.
624{634 (2009).
83. Norman, K.L. Geq (game engagement/experience
questionnaire): a review of two papers", Interacting
with Computers, 25(4), pp. 278{283 (2013).
84. Von Ahn, L. and Dabbish, L. Designing games with
a purpose", Commun. ACM, 51(8), pp. 58{67 (2008).