A Software Agent Illustrating Some Features of an Illusionist Account of Consciousness

1.Buck Shlegeris wrote the software agent, and the appendix containing notes for programmers who might want to review the agent’s code. I (Luke) suggested the specific theories to attempt implementing, and wrote the rest of the report. The main body of the report is written in my voice because it discusses the origins and evolution of my personal views and intuitions, which don’t necessarily match Buck’s views and intuitions.

2.For a quick introduction to some functionalist theories of consciousness, see Weisberg (2014), chs. 6-8. On illusionist theories of consciousness in particular, see Appendix F of my earlier report on consciousness and moral patienthood, and also Frankish (2016a).

3.I make this critique of functionalist theories of consciousness in Appendix B of my report on consciousness and moral patienthood. Below I quote some additional examples of (roughly) this objection being made.

During a February 2017 “Ask Me Anything” session on Reddit.com, David Chalmers made similar complaints about illusionist theories of consciousness. Below, I link to each comment from which I quote. I have also reformatted Chalmers’ replies a bit, for clarity:

[1] I think [Dennett’s] basic view, that consciousness is an illusion, is a really important one to pursue. If I have any complaint it would be that he hasn’t pursued it strongly and deeply enough in recent years.

[2] Regarding the paradox of phenomenal judgment: I agree the key is finding a functional explanations of why we make judgments such as “I am conscious”, “consciousness is mysterious”, “there’s a hard problem of consciousness over and above the easy problems”, and so on. I tried to give the beginnings of such an explanation at a couple of points in The Conscious Mind, but it wasn’t well-developed… Illusionists like Dennett, Humphrey, Graziano, Drescher, and others have also tried giving elements of such a story, but usually also in a very sketchy way that doesn’t seem fully adequate to the behavior that needs to be explained. Still I think there is a real research program here that philosophers and scientists of all stripes ought to be able to buy into… It’s an under-researched area at the moment and I hope it gets a lot more attention in the coming years. I’m hoping to return soon to this area myself.

In response to Frankish (2016a)’s defense of illusionism about consciousness, Marinsek & Gazzaniga (2016) write:

One major limitation of [illusionism as described by Frankish] is that it does not offer any mechanisms for how the illusion of phenomenal feelings works. As anyone who has seen a magic trick knows, it’s quite easy to say that the trick is an illusion and not the result of magical forces. It is much, much harder to explain how the illusion was created. Illusionism can be a useful theory if mechanisms are put forth that explain how the brain creates an illusion of phenomenal feelings…

…phenomenal consciousness may not be the product of one grand illusion. Instead, phenomenal consciousness may be the result of multiple ‘modular illusions’. That is, different phenomenal feelings may arise from the limitations or distortions of different cognitive modules or networks… Illusionism therefore may not have to account for one grand illusion, but for many ‘modular illusions’ that each have their own neural mechanisms.

4.On phenomenal judgments, see e.g. Chalmers (1996), pp. 184-186 and 288-292; Molyneux (2012); Graziano (2016).

On the properties of our sensory qualia, see e.g. Clark (1993); O’Regan (2011).

On the apparent unity of conscious experience, see e.g. Bayne (2010); Bennett & Hill (2014); Brook & Raymont (2017).

5.Other motivations for this project included (1) the intuition that to understand something better, it is often helpful to try to build it, and (2) a desire to test the intuition that many theories of consciousness seem as though they’d be satisfied by a relatively simple computer program.

On (1): A commonly-quoted version of this dictum was found on Richard Feynman’s blackboard upon his death: “What I cannot create, I do not understand.” Below are some examples of scholars expressing similar sentiments in the context of consciousness studies.

Molyneux (2012):

…Instead of attempting to solve what appears unsolvable, an alternative reaction is to investigate why the problem seems so hard. In this way, Minsky (1965) hoped, we might at least explain why we are confused. Since a good way to explain something is often to build it, a good way to understand our confusion [about consciousness] may be to build a robot that thinks the way we do… I hope to show how, by attempting to build a smart self-reflective machine with intelligence comparable to our own, a robot with its own hard problem, one that resembles the problem of consciousness, may emerge.

Graziano (2016):

One useful way to introduce [Graziano’s theory of consciousness] is through the hypothetical challenge of building a robot that asserts it is subjectively aware of an object and describes its awareness in the same ways that we do.

Reggia (2013):

What would be the value of studying consciousness with machines? There are two main answers to this question. First and foremost would be to improve our scientific understanding of the nature of consciousness. At the present time, our understanding of how a physical system such as the human brain can support the subjective experiences that are at the core of a conscious mind are largely pre-scientific… Individuals working on artificial consciousness… [observe] that computer models of specific aspects of consciousness (simulated consciousness) may prove to be useful in advancing our understanding of conscious information processing, just as computer models are useful in many other fields.

On (2): George Rey and others have suggested that it would be fairly simple to write a computer program that seems to satisfy many popular theories of consciousness. See e.g. Rey (1983):

…it seems to me to be entirely feasible… to render an existing computing machine intentional by providing it with a program that would include the following:

1. The alphabet, formation, and transformation rules for quantified modal logic (the system’s “language of thought”).

2. The axioms for your favorite inductive logic and/or abductive system of hypotheses, with a “reasonable” function for selecting among them on the basis of given input.

3. The axioms of your favorite decision theory, and some set of basic preferences.

4. Mechanical inputs, via sensory transducers, for Clauses 2 and 3.

5. Mechanical connections that permit the machine to realize its outputs (e.g., its “most preferred” basic act descriptions).

Any computer that functioned according to such a program would, I submit, realize significant Rational Regularities, complete with intensionality. Notice, for example, that it would be entirely appropriate — and probably unavoidable — for us to explain and predict its behavior and internal states on the basis of those regularities. It would be entirely reasonable, that is to say, for us to adopt toward it what Dennett (1971) has called the “intentional stance.”

…However clever a machine programmed with Clauses 1-5 might become, counting thereby as a thinking thing, surely it would not also count thereby as conscious. The program is just far too trivial. Moreover, we are already familiar with systems satisfying at least Clauses 1-5 that we also emphatically deny are conscious: there are all those unconscious neurotic systems postulated in so many of us by Freud, and all those surprisingly intelligent, but still unconscious, subsystems for perception and language postulated in us by contemporary cognitive psychology. (Some evidence of the cognitive richness of unconscious processing is provided by the interesting review of such material in Nisbett & Wilson, 1977, but especially by such psycholinguistic experiments as that by Lackner & Garrett, 1973, in which subliminal linguistic material provided to one ear biased subjects in their understanding of ambiguous sentences provided to the other ear.) In all of these cases we are, I submit, quite reasonably led to ascribe beliefs, preferences, and sometimes highly elaborate thought processes to a system on the basis of the Rational Regularities, despite the fact that the systems involved are often not the least bit “conscious” of any such mental activity at all. It is impossible to imagine these psychological theories getting anywhere without the ascription of unconscious content — and it is equally difficult to imagine any animals getting anywhere without the exploitation of it. Whatever consciousness will turn out to be, it will pretty certainly need to be distinguished from the thought processes we ascribe on the basis of the rational regularities.

How easily this point can be forgotten, neglected, or missed altogether is evidenced by the sorts of proposals about the nature of consciousness one finds in some of the recent psychobiological literature. The following seem to be representative:

Consciousness is usually defined by the ability: (1) to appreciate sensory information; (2) to react critically to it with thoughts or movements; (3) to permit the accumulation of memory traces. (Moruzzi, 1966)

Perceptions, memories, anticipatory organization, a combination of these factors into learning — all imply rudimentary consciousness. (Knapp, 1976)

Modern views… regard human conscious activity as consisting of a number of major components. These include the reception and processing (recoding) of information, with the selection of its most important elements and retention of the experience thus gained in the memory; enunciation of the task or formulation of an intention, with the preservation of the corresponding modes of activity, the creation of a pattern or model of the required action, and production of the appropriate program (plan) to control the selection of necessary actions; and finally the comparison of the results of the action with the original intention … with correction of the mistakes made. (Luria, 1978)

Consciousness is a process in which information about multiple individual modalities of sensation and perception is combined into a unified, multidimensional representation of the state of the system and its environment and is integrated with information about memories and the needs of the organism, generating emotional reactions and programs of behavior to adjust the organism to its environment. (John, 1976)

What I find astonishing about such proposals is that they are all more-or-less satisfiable by almost any information-processing system, for precisely what modern computational machinery is designed to do is to receive, process, unify, and retain information; create (or “call”) patterns, models, and subroutines to control its activity; and, by all means, to compare the results of its action with its original intention in order to adjust its behavior to its environment. This latter process is exactly what the “feedback” that Wiener (1954) built into his homing rocket was for! Certainly, most of the descriptions in these proposals are satisfied by any recent game-playing program (see, e.g., Berliner, 1980). And if it’s genuine “modalities,” “thoughts,” “intentions,” “perceptions,” or “representations” that are wanted, then, as I’ve argued, supplementing the program with Clauses 1-5 will suffice, but without rendering anything a whit more conscious.

White (1991), ch. 6, summed up Rey’s point like so:

…a survey of recent characterizations of consciousness by philosophers and psychologists reveals that most or all characterizations would be satisfied by information-processing devices that either exist now or would be trivial extensions of devices that exist.

See also Rey (1995, 2016) and the “small network argument” of Herzog et al. (2007).

6.The software agent would better-illustrate an illusionist approach if we could say that it is clear that it “mistakenly judges that it is impossible for an agent to be mistaken about its own experiences,” but we decided not to put in the extra work required for the agent to clearly satisfy that criterion.

7.In this sense, this project is similar in motivation to many other “machine consciousness” research projects (Cold Spring Harbor Laboratories 2001; Gamez 2008; Reggia 2013; Aleksander 2017). Arguably, the major distinguishing characteristic of the present project is merely its particular (illusionism-motivated) selection of theories to attempt to instantiate.

8.This number is analogous to the gensym name is Drescher’s “qualia as gensyms” account (Drescher 2006, ch. 2).

9.Technically, the agent’s “yes” reply is “necessarily true,” and its “no” reply is “necessarily false.” The other possible replies from Z3 are equivalent to “Both that statement and its negation are possible” (which I translate as “I don’t know”) and “The axioms I was passed contain a contradiction.” Note that for the agent, 0-255 acts as a color wheel, in the sense that the difference between 255 and 0 is 1 rather than 256.

10.Chalmers (1990):

Very briefly, here is what I believe to be the correct account of why we think we are conscious, and why it seems like a mystery. The basic notion is that of pattern processing. This is one of the things that the brain does best. It can take raw physical data, usually from the environment but even from the brain itself, and extract patterns from these. In particular, it can discriminate on the basis of patterns. The original patterns are in the environment, but they are transformed on their path through neural circuits, until they are represented as quite different patterns in the cerebral cortex. This process can also be represented as information flow (not surprisingly), from the environment into the brain. The key point is that once the information flow has reached the central processing portions for the brain, further brain function is not sensitive to the original raw data, but only to the pattern (to the information!) which is embodied in the neural structure.

Consider color perception, for instance. Originally, a spectral envelope of light-wavelengths impinges upon our eyes. Immediately, some distinctions are collapsed, and some pattern is processed. Three different kinds of cones abstract out information about how much light is present in various overlapping wavelength-ranges. This information travels down the optic nerve (as a physical pattern, of course), where it gets further transformed by neural processing into an abstraction about how much intensity is present on what we call the red-green, yellow-blue, and achromatic scales. What happens after this is poorly-understood, but there is no doubt that by the time the central processing region is reached, the pattern is very much transformed, and the information that remains is only an abstraction of certain aspects of the original data.

Anyway, here is why color perception seems strange. In terms of further processing, we are sensitive not to the original data, not even directly to the physical structure of the neural system, but only to the patterns which the system embodies, to the information it contains. It is a matter of access. When our linguistic system (to be homuncular about things) wants to make verbal reports, it cannot get access to the original data; it does not even have direct access to neural structure. It is sensitive only to pattern. Thus, we know that we can make distinctions between certain wavelength distributions, but we do not know how we do it. We’ve lost access to the original wavelengths – we certainly cannot say “yes, that patch is saturated with 500-600 nm reflections”. And we do not have access to our neural structure, so we cannot say “yes, that’s a 50 Hz spiking frequency”. It is a distinction that we are able to make, but only on the basis of pattern. We can merely say “Yes, that looks different from that.” When asked “How are they different?”, all we can say is “Well, that one’s red, and that one’s green”. We have access to nothing more – we can simply make raw distinctions based on pattern – and it seems very strange.

So this is why conscious experience seems strange. We are able to make distinctions, but we have direct access neither to the sources of those distinctions, or to how we make the distinctions. The distinctions are based purely on the information that is processed. Incidentally, it seems that the more abstract the information-processing – that is, the more that distinctions are collapsed, and information recoded – the stranger the conscious experience seems. Shape- perception, for instance, strikes us as relatively non-strange; the visual system is extremely good at preserving shape information through its neural pathways. Color and taste are strange indeed, and the processing of both seems to involve a considerable amount of recoding.

The story for “internal perception” is exactly the same. When we reflect on our thoughts, information makes its way from one part of the brain to another, and perhaps eventually to our speech center. It is to only certain abstract features of brain structure that the process is sensitive. (One might imagine that if somehow reflection could be sensitive to every last detail of brain structure, it would seem very different.) Again, we can perceive only via pattern, via information. The brute, seemingly non-concrete distinctions thus entailed are extremely difficult for us to understand, and to articulate. That is why consciousness seems strange, and that is why the debate over the Mind-Body Problem has raged for thousands of years.

Chalmers revised this account somewhat in Chalmers (1996), pp. 184-186 and 288-292.

Another version of the basic idea is Drescher’s “qualia as gensyms” account (Drescher 2006, ch. 2).

A related idea, cast in terms of neural networks, can be found in Loosemore (2012), which Tomasik (2014) summarizes like this:

Loosemore presents what I consider a biologically plausible sketch of connectionist concept networks in which a concept’s meaning is assessed based on related concepts. For instance, “chair” activates “legs”, “back”, “seat”, “sitting”, “furniture”, etc. (p. 294). As we imagine lower-level concepts, the associations that get activated become more primitive. At the most primitive level, we could ask for the meaning of something like “red”. Since our “red” concept node connects directly to sensory inputs, we can’t decompose “red” into further understandable concepts. Instead, we “bottom out” and declare “red” to be basic and ineffable. But our concept-analysis machinery still claims that “red” is something — namely, some additional property of experience. This leads us to believe in qualia as “extra” properties that aren’t reducible.

11.To elicit this judgment, we can ask the agent a question of the form “For all 2 agents and one hue, is it true that the the experience of agent 1 and that hue is the same as the experience of agent 2 and that hue?” The reasoning system replies: “Both that and its negation are possible.” For more detail on why this judgment is returned, see the code. Note that for technical reasons the agent doesn’t “believe” all the statements made above (which I gave for providing the right intuition), but instead deduces the possibility of inverted or rotated spectra from the mere belief that other agents experience the same relations between different colors as itself.

12.In my earlier report, I mentioned the present project in section 5.1:

…I’d like to work with a more experienced programmer to sketch a toy program that I think might be conscious if elaborated, coded fully, and run. Then, I’d like to adjust the details of its programming so that it more closely matches my own first-person data and the data gained from others’ self-reports of conscious experience (e.g. in experiments and in brain damage cases)… We have begun to collaborate with a programmer on such a project, but we’re not sure how much effort we will put into it at this time.

I also outlined what a more ambitious version of this project might look like, in section 6.2.4.