Visual creativity through concept combination using quantum cognitive models

Document Type : Article

Authors

Complex Systems (Chaos) Laboratory, School of Computer Engineering, Iran University of Science and Technology, Tehran, Postal Code: 16846-13114, Iran

Abstract

Computational creativity modeling, including concept combination, enables us to foster deeper abilities of AI agents. Although concept combination has been addressed in a lot of computational creativity studies, findings show incompatibility amongst empirical data of concept combination and the results of the used methods. In addition, even though recent neuroscientific studies show the crucial impact of retrieving concepts’ relations explicitly stored in episodic memory, it has been underestimated in modeling creative processes. In this paper, a quantum cognition-based approach is used to more effectively consider the context and resolve logical inconsistencies. Also, episodic memory is leveraged as the basis for the concept combination modeling process based on the created context. The result of the proposed process is a set of meaningful concepts and expressions as a combination of stimuli and related episodes which are used to depict a visual collage as an image. The significant improvement in the quality of results in comparison with the existing methods suggests that quantum-like modeling can be considered as the foundation for developing AI agents capable of creating artistic images or assisting a person during a creative process.

Keywords

References

References:
1. Russell, S. and Norvig, P., Artificial Intelligence: A Modern Approach, 3rd Ed., Person, pp. 1-33 (2002).
2. Boden, M.A., In The Creative Mind: Myths and Mechanisms, Routledge, pp. 1-10 (2004). DOI: 10.4324/9780203508527.
3. Wiggins, G.A. "Searching for computational creativity", New Gener Comput, 24(3), pp. 209-222 (Sep. 2006). DOI: 10.1007/BF03037332.
4. Colton, S. and Wiggins, G.A. "Computational Creativity: The Final Frontier?", The Final Frontier, Association for Computational Creativity (2013).
5. Ullman, M.T. "Contributions of memory circuits to language", The Declarative/Procedural Model, Cognition, 92(1), pp. 231-270 (May 2004). DOI: 10.1016/j.cognition.2003.10.008.
6. G.A. Radvansky, Human Memory, 4th Edition, ISBN: 9780367252922, Published March 31, 2021 by Routledge.
7. Tulving, E. "Episodic and semantic memory: Where should we go from here?" Behavioral and Brain Sciences, 9(3), pp. 573-577 (1986).
8. Willingham, D.B., Nissen, M.J., and Bullemer, P. "On the development of procedural knowledge.", Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), pp. 1047-1060 (1989).
9. Razumnikova, O.M. "Divergent versus convergent thinking", In Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship, E.G. Carayannis, Ed., New York, NY: Springer, pp. 546-552 (2013). DOI: 10.1007/978-1-4614-3858-8 362.
10. Gilhooly, K.J., Fioratou, E., Anthony, S.H., et al.  Divergent thinking: Strategies and executive involvement in generating novel uses for familiar objects", British Journal of Psychology, 98(4), pp. 611-625 (2007). DOI: https://doi.org/10.1111/j.2044-8295. 2007.tb00467.x.
11. Wiggins, G.A. and Bhattacharya, J. "Mind the gap: an attempt to bridge computational and neuroscientific approaches to study creativity", Front Hum Neurosci, 8, pp. 540-540 (Jul. 2014). DOI: 10.3389/fnhum.2014.00540.
12. Yee, E., Chrysikou, E.G., and Thompson-Schill, S.L.  Semantic memory", The Oxford Handbook of Cognitive Neuroscience, 1, Core topics, In Oxford library of psychology, New York, NY, US: Oxford University Press, pp. 353-374 (2014).
13. Shalley, C.E., Zhou, J., and Oldham, G.R., "The effects of personal and contextual characteristics on creativity: Where should we go from here?", Journal of Management, 30(6), pp. 933-958 (Dec. 2004). DOI: 10.1016/j.jm.2004.06.007.
14. Dietrich, A. and Kanso, R. "A review of EEG, ERP, and neuroimaging studies of creativity and insight", Psychol Bull, 136(5), pp. 822-848 (Sep. 2010). DOI: 10.1037/a0019749.
15. Addis, D.R., Pan, L., Musicaro, R., et al. "Divergent thinking and constructing episodic simulations", Memory, 24(1), pp. 89-97 (2016). DOI: 10.1080/09658211.2014.985591.
16. Schacter, D.L. and Madore, K.P. "Remembering thepast and  magining the future: Identifying and enhancing the contribution of episodic memory", Memory Studies, 9(3), pp. 245-255 (Jul. 2016). DOI: 10.1177/1750698016645230.
17. Schacter, D.L., Addis, D.R., Hassabis, D., et al. "The future of memory: Remembering, imagining, and the brain", Neuron, 76(4), pp. 677-694 (Nov. 2012). DOI: 10.1016/j.neuron.2012.11.001.
18. Duff, M.C., Kurczek, J., Rubin, R., et al. "Hippocampal amnesia disrupts creative thinking", Hippocampus, 23(12), pp. 1143-1149 (2013). DOI: https://doi.org/10.1002/hipo.22208.
19. Ellamil, M., Dobson, C., Beeman, M., et al. "Evaluative and generative modes of thought during the creative process", NeuroImage, 59(2), pp. 1783-1794 (Jan. 2012). DOI: 10.1016/j.neuroimage.2011.08.008.
20. Benedek, M., Jauk, E., Fink, A., et al. "To create or to recall? Neural mechanisms underlying the generation of creative new ideas", NeuroImage, 88, pp. 125-133 (Mar. 2014). DOI: 10.1016/j.neuroimage.2013.11.021.
21. Kotseruba, I. and Tsotsos, J.K. "A review of 40 years of cognitive architecture research: Core cognitive abilities and practical applications", arXiv:1610.08602 [cs], Jan. 2018, Accessed: Feb. 27 (2021). http://arxiv.org/abs/1610..08602.
22. Bruza, P.D., Wang, Z., and Busemeyer, J.R. "Quantum cognition: a new theoretical approach to psychology", Trends in Cognitive Sciences, 19(7), pp. 383-393 (Jul. 2015). DOI: 10.1016/j.tics.2015.05.001.
23. Osherson, D.N. and Smith, E.E. "On the adequacy of prototype theory as a theory of concepts", Cognition, 9(1), pp. 35-58 (Jan. 1981). DOI: 10.1016/0010-0277(81)90013-5.
24. Hampton, J.A. "Disjunction of natural concepts", Memory and Cognition, 16(6), pp. 579-591 (Nov. 1988). DOI: 10.3758/bf03197059.
25. Hampton, J.A. "Conceptual combination: Conjunction and negation of natural concepts", Memory and Cognition, 25(6), pp. 888-909 (Nov. 1997). DOI: 10.3758/bf03211333.
26. Gabora, L. and Aerts, D. "Contextualizing concepts using a mathematical generalization of the quantum formalism", Journal of Experimental and Theoretical Artificial Intelligence, 14(4), pp. 327-358 (Oct. 2002). DOI: 10.1080/09528130210162253.
27. Tversky, A. and Kahneman, D. "Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment", Psychological Review, 90(4), pp. 293-315 (1983). DOI: 10.1037/0033-295x.90.4.293.
28. Pothos, E.M. and Busemeyer, J.R. "Can quantum probability provide a new direction for cognitive modeling?" Behavioral and Brain Sciences, 36(3), pp. 255- 274 (May 2013). DOI: 10.1017/s0140525x12001525.
29. Aerts, D. and Gabora, L. "A theory of concepts and their combinations I", Kybernetes, 34(1/2), pp. 167- 191 (Jan. 2005). DOI: 10.1108/03684920510575799.
30. Ventura, D. "The computational creativity complex", Computational Creativity Research: Towards Creative Machines, T.R. Besold, M. Schorlemmer, and A. Smaill, Eds., In Atlantis Thinking Machines, 7, Paris: Atlantis Press, pp. 65-92 (2015). DOI: 10.2991/978-94-6239-085-0.
31. Kiss, G.R., Armstrong, C., Milroy, R., et al. "An associative thesaurus of English and its computer analysis", The Computer and Literary Studies, A.J. Aitkin, R.W. Bailey, and N. Hamilton-Smith, Eds., Edinburgh, UK: University Press (1973).
32. Nelson, D.L., McEvoy, C.L., and Schreiber T.A. "The University of South Florida free association, rhyme, and word fragment norms", Behavior Research Methods, Instruments, and Computers, 36(3), pp. 402-407 (Aug. 2004). DOI: 10.3758/BF03195588.
33. Miller, G.A., WordNet: An Electronic Lexical Database, Cambridge, Mass: A Bradford Book (1998).
34. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al. "Generative adversarial networks", Commun. ACM, 63(11), pp. 139-144 (Oct. 2020). DOI: 10.1145/3422622.
35. Tao, M., Tang, H., Wu, F., et al. "DF-GAN: A simple and effective baseline for text-to-image synthesis", arXiv (2022). DOI: 10.48550/arXiv.2008.05865.
36. Xu, T., Zhang, P., Huang, Q, et al. "AttnGAN: Fine grained text to image generation with attentional generative adversarial networks", In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA: IEEE, pp. 1316-1324 (Jun. 2018). DOI:10.1109/CVPR.2018.00143.
37. Ye, H., Yang, X., Takac, M., et al. "Improving textto-image synthesis using contrastive learning", arXiv (2021). DOI: 10.48550/arXiv.2107.02423.
38. Zhang, H., Koh, J.Y., Baldridge, J., et al. "Crossmodal contrastive learning for text-to-image generation", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 833-842 (May 2023). https://openaccess.thecvf.com/content/ CVPR2021/html/Zhang Cross-Modal Contrastive Learning for Text-to-Image Generation CVPR 2021 paper.html.
39. Zhu, M., Pan, P., Chen, W., et al. "DM-GAN: Dynamic memory generative adversarial networks for text-to-image synthesis", In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA: IEEE, pp. 5802-5810 (2019).DOI: 10.1109/CVPR.2019.00595.
40. Ramesh, A., Pavlov, M., Goh, G., et al. "Zeroshot text-to-image generation", In Proceedings of the 38th International Conference on Machine Learning, PMLR, pp. 8821-8831 (2023). https://proceedings.mlr.press/v139/ramesh21a.html.
41. Oord, A.v.d., Vinyals, O., and Kavukcuoglu, K. "Neural discrete representation learning", Advances in Neural Information Processing Systems, Curran Associates, Inc. (2017). https://proceedings.neurips. cc/paper/2017/hash/7a98af17e63a0ac09ce2e9 6d03992fbc-Abstract.html.
42. Razavi, A., Oord, A.v.d., and Vinyals, O. "Generating diverse high-fidelity images with VQ-VAE-2", Advances in Neural Information Processing Systems, Curran Associates, Inc. (2019). https://proceedings.neurips.cc/paper/2019/hash/5f8e 2fa1718d1bbcadf1cd9c7a54fb8c-Abstract.html.
43. Preechakul, K., Chatthee, N., Wizadwongsa, S., et al. "Diffusion autoencoders: Toward a meaningful and decodable representation", Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10619-10629 (2022). https://openaccess.thecvf.com/content/CVPR2022/ html/Preechakul Diffusion Autoencoders Toward a Meaningful and Decodable Representation CVPR 2022 paper.html.
44. Rombach, R., Blattmann, A., Lorenz, D., et al. "High-resolution image synthesis with latent diffusion models", Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684-10695 (2022). https://openaccess.thecvf.com/content/CVPR2022/ html/Rombach High-Resolution Image Synthesis With Latent Diffusion Models CVPR 2022 paper.html.
45. Esser, P., Rombach, R., and Ommer, B. "Taming transformers for high-resolution image synthesis", Presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873-12883 (2021). https://openaccess.thecvf.com/content/CVPR2021/ html/Esser Taming Transformers for High- Resolution Image Synthesis CVPR 2021 paper.html?ref=https://githubhelp.com.
46. Radford, A., Kim, JW., Hallacy, C., et al. "Learning transferable visual models from natural language supervision", In Proceedings of the 38th International Conference on Machine Learning, PMLR, pp. 8748- 8763 (2021). https://proceedings.mlr.press/v139/radford21a.html.
47. Nichol, A., Dhariwal, P., Ramesh, A., et al. "GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models", arXiv, (Mar. 2022). DOI: 10.48550/arXiv.2112.10741.
48. Crowson K. "CLIP guided diffusion HQ 256x256" (Apr. 2023). https://colab.research.google.com/drive/ 12a Wrfi2 gwwAuN3VvMTwVMz9TfqctNj, accessed.
49. Crowson K., "CLIP guided diffusion HQ 512x512", (2023). https://colab.research.google.com/drive /1V66mUeJbXrTuQITvJunvnWVn96FEbSI3.
50. Wang, Z., Liu, W., He, Q., et al. "CLIP-GEN: language-free trraining of a trext-to-image generator with CLIP", arXiv, pp. 1-15 (Mar. 2022). DOI: 10.48550/arXiv.2203.00386.
51. Zhou, Y., Zhang, R., Chen, C., et al. "LAFITE: Towards language-free training for trext-to-image generation", arXiv, pp. 1-17 (Mar. 2022). DOI: 10.48550/arXiv.2111.13792.
52. Sauer, A., Karras, T., Laine, S., et al. "StyleGAN-T: Unlocking the power of GANs for fast large-scale trextto- image synthesis", arXiv, pp. 1-13 (Jan. 2023). DOI: 10.48550/arXiv.2301.09515.
53. Sauer, A., Schwarz, K., and Geiger, A. "StyleGAN-XL: Scaling styleGAN to large diverse datasets", In ACM SIGGRAPH 2022 Conference Proceedings, New York, NY, USA: Association for Computing Machinery, pp. 1-10 (Jul. 2022). DOI: 10.1145/3528233.3530738.
54. Tao, M., Bao, B.K., Tang, H., et al. "GALIP: Generative adversarial CLIPs for trext-to-image synthesis", arXiv, pp. 1-11 (Jan. 2023). DOI: 10.48550/arXiv.2301.12959.
55. Rose, S., Engel, D., Cramer, N., et al. "Automatic keyword extraction from individual documents", In Text Mining, John Wiley and Sons, Ltd, pp. 1-20 (2010). DOI: 10.1002/9780470689646.ch1.
56. Barros, J., Toffano, Z., Meguebli, Y., et al. "Contextual query using bell tests", In Quantum Interaction, H. Atmanspacher, E. Haven, K. Kitto, and D. Raine, Eds., In Lecture Notes in Computer Science, 8369. Berlin, Heidelberg: Springer, pp. 110-121 (2014). DOI: 10.1007/978-3-642-54943-4 10.
57. Blei, D.M., Ng, A.Y., and Jordan, M.I. "Latent dirichlet allocation", J. Mach. Learn. Res., 3, pp. 993- 1022 (Mar. 2003).
58. "fastText." https://fasttext.cc/index.html (accessed Feb. 26, 2023).
59. "Custom Search JSON API." URL: https://developers. google.com/custom-searc h/v1/overview accessed Feb. 27 (2023).
60. "Google-Images-Search." https://libraries.io/pypi/Google-Images-Search (accessed Feb. 27, 2023).
61. "Opencv-python." https://libraries.io/pypi/opencvpython accessed Feb. 27 (2023).
62. Roder, M., Both, A., and Hinneburg, A. "Exploring the space of topic coherence measures", In Proceedings of the 8th ACM International Conference on Web Search and Data Mining, In WSDM', 15. New York, NY, USA: Association for Computing Machinery, pp. 399-408 (Feb. 2015). DOI: 10.1145/2684822.2685324.
63. Ventura, D. "Autonomous intentionality in computationally creative systems", in Computational Creativity: The Philosophy and Engineering of Autonomously Creative Systems, T. Veale and F. A. Cardoso, Eds., in Computational Synthesis and Creative Systems. Cham: Springer International Publishing, pp. 49-69 (2019). DOI: 10.1007/978-3-319-43610-4 3.
Scientia Iranica
Volume 31, Issue 2 - Serial Number 2
Transactions on Computer Science & Engineering and Electrical Engineering (D)
January and February 2024
Pages 118-135

Files

History

  • Receive Date: 24 November 2022
  • Revise Date: 19 May 2023
  • Accept Date: 15 August 2023

Share

How to cite

Statistics